Getting Started
===============

Welcome to pollywog! This guide will help you install the library, understand the basics, run your first calculation, and explore further resources.

.. note::
   **New to Python?** If you're a geologist or mining engineer with limited programming experience,
   see :doc:`geologist_tutorial` for a comprehensive step-by-step introduction designed specifically for you.

What You Need to Know
----------------------

Before using pollywog, it's helpful to understand:

- **Python basics**: Functions, lists, dictionaries, and imports
- **Leapfrog concepts**: Block models, drillholes, calculations
- **.lfcalc files**: Leapfrog calculation set files

You don't need to be a Python expert—pollywog is designed to be accessible to geologists and mining engineers with basic programming knowledge.

Installation
------------

Install from PyPI
~~~~~~~~~~~~~~~~~

The easiest way to install pollywog:

.. code-block:: bash

    pip install lf_pollywog

Install with Optional Dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For machine learning model conversion (scikit-learn):

.. code-block:: bash

    pip install lf_pollywog[conversion]

For development (includes testing tools):

.. code-block:: bash

    pip install lf_pollywog[dev]

Install from GitHub
~~~~~~~~~~~~~~~~~~~

To get the latest development version:

.. code-block:: bash

    pip install git+https://github.com/endarthur/pollywog.git

Verify Installation
~~~~~~~~~~~~~~~~~~~

Check that pollywog is installed correctly:

.. code-block:: python

    import pollywog as pw
    print(pw.__version__)  # Should print version number

Try in Your Browser
~~~~~~~~~~~~~~~~~~~

Don't want to install yet? Try pollywog in your browser using JupyterLite:

    https://endarthur.github.io/pollyweb

.. note::
    JupyterLite saves files in browser memory—download your work regularly to avoid losing it!

Basic Concepts
--------------

CalcSet
~~~~~~~

A ``CalcSet`` is a collection of calculation items that can be exported to a ``.lfcalc`` file:

.. code-block:: python

    from pollywog.core import CalcSet, Number

    calcset = CalcSet([
        Number("Au_clean", "clamp([Au], 0)"),
        Number("Au_log", "log([Au_clean] + 1e-6)"),
    ])

Number
~~~~~~

``Number`` represents a numeric calculation (integers or floats):

.. code-block:: python

    from pollywog.core import Number

    # Simple calculation
    grade_calc = Number("Au_final", "[Au] * 0.95")

    # With comment
    grade_calc = Number(
        "Au_final",
        "[Au] * 0.95",
        comment_equation="Apply 5% dilution factor"
    )

Category
~~~~~~~~

``Category`` represents a categorical/text calculation:

.. code-block:: python

    from pollywog.core import Category, If

    # Categorical output
    ore_type = Category(name="material_class", expression=[
        If("[Au] >= 0.5", "'ore'", "'waste'")
    ])

Variable
~~~~~~~~

``Variable`` represents intermediate calculations that are visible in Leapfrog's calculator UI but **NOT available outside the calculator** (can't visualize in 3D, can't export, can't use in other Leapfrog tools).

Use Variables for intermediate steps you need for calculations but won't visualize or export:

.. code-block:: python

    from pollywog.core import CalcSet, Variable, Number

    calcset = CalcSet([
        # Intermediate cleaning steps (calculator only)
        Variable("Au_clean", "clamp([Au], 0)"),
        Variable("Au_capped", "clamp([Au_clean], 0, 100)"),

        # Final result (available everywhere in Leapfrog)
        Number("Au_final", "[Au_capped] * 0.95",
               comment_equation="Prepared for kriging")
    ])

**Why use Variables?**

Variables keep your Leapfrog interface clean by hiding intermediate calculations that users don't need to see or export. They're visible in the calculator (so you can reference them and verify logic), but they won't clutter your block model variable list or visualization options.

**When to use Variable vs Number vs Category:**

- **Variable**: Intermediate steps, won't visualize/export
- **Number**: Final numeric outputs you'll visualize or export
- **Category**: Final categorical outputs you'll visualize or export

.. mermaid::

   flowchart TD
      Q1{What's the output?}
      Q1 -->|Numeric| Q2{Will you visualize it?}
      Q1 -->|Text/Category| Q3{Will you visualize it?}
      Q1 -->|Boolean/Filter| F[Filter]

      Q2 -->|Yes| N[Number]
      Q2 -->|No - just intermediate| V1[Variable]

      Q3 -->|Yes| C[Category]
      Q3 -->|No - just intermediate| V2[Variable]

      style N fill:#4a90e2,stroke:#333,stroke-width:2px,color:#fff
      style C fill:#f4a261,stroke:#333,stroke-width:2px,color:#fff
      style V1 fill:#999,stroke:#333,stroke-width:2px,color:#fff
      style V2 fill:#999,stroke:#333,stroke-width:2px,color:#fff
      style F fill:#9b59b6,stroke:#333,stroke-width:2px,color:#fff

Filter
~~~~~~

``Filter`` represents boolean conditions that restrict which data is included in calculations. Filters evaluate to true/false and are used to exclude data from processing.

.. code-block:: python

    from pollywog.core import CalcSet, Filter, Number

    calcset = CalcSet([
        # Only include high-grade samples
        Filter("is_ore", "[Au] > 0.5"),

        # Only include complete data
        Filter("has_density", "is_normal([density])"),

        # Calculations on filtered data
        Number("ore_tonnage", "[volume] * [density]")
    ])

Filters are less commonly used in block models (where you typically calculate everything and classify later), but are useful for drillhole data processing and conditional workflows.

Expressions and Syntax
~~~~~~~~~~~~~~~~~~~~~~~

In pollywog, expressions use the same syntax as Leapfrog's calculator. Variables are referenced with square brackets ``[variable_name]``, and you can use mathematical operations and functions.

Quick example:

.. code-block:: python

    Number("precious_metals", "[Au] + [Ag]")
    Number("density_calc", "[block_density] * [block_volume]")

.. seealso::
   For complete expression syntax, functions, and operators, see :doc:`expression_syntax`

Understanding the Expression Parameter
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``expression`` parameter accepts either a **string** or a **list**:

- **Use a string** for simple expressions (recommended for most cases):

  .. code-block:: python

      Number("doubled", "[x] * 2")
      Category("rock_type", "'granite'")

- **Use a list** when including ``If`` objects for conditional logic:

  .. code-block:: python

      Number("result", [If("[x] > 0", "[x]", "0")])

**Why can expression be a list?**

``If`` statements are separate Python objects that cannot be embedded in expression strings.
Since the Leapfrog calculation format supports conditional logic, pollywog needs to handle
these ``If`` objects alongside string expressions. That's why the parameter accepts lists.

For convenience, single strings are automatically wrapped in a list internally, so you don't
need to write ``expression=["[x] * 2"]`` - just use ``expression="[x] * 2"`` or the even
simpler positional form: ``Number("doubled", "[x] * 2")``.

**If vs IfRow:**

``If`` accepts condition-value pairs as tuples (recommended for most users):

.. code-block:: python

    If([
        ("[Au] > 1", "[Au] * 1.1"),
        ("[Au] <= 1", "[Au] * 0.9")
    ], otherwise=["[Au]"])

``IfRow`` is a lower-level class representing a single condition-value pair, used internally by ``If``. Most users won't need to use ``IfRow`` directly - stick with tuple notation in ``If`` for clarity.

Your First Calculation
-----------------------

Let's create a simple calculation set step by step:

Step 1: Import pollywog
~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from pollywog.core import CalcSet, Number

Step 2: Create Calculations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    # Create individual calculations
    au_clean = Number(
        "Au_clean",
        "clamp([Au], 0)",
        comment_equation="Remove negative values"
    )

    au_scaled = Number(
        "Au_scaled",
        "[Au_clean] * 0.95",
        comment_equation="Apply 95% factor"
    )

Step 3: Build a CalcSet
~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    # Combine into a CalcSet
    calcset = CalcSet([au_clean, au_scaled])

Step 4: Export to Leapfrog
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    # Export to .lfcalc file
    calcset.to_lfcalc("my_calculations.lfcalc")

    print(f"Exported {len(calcset.items)} calculations")

.. note::
    In JupyterLite and Jupyter Notebooks, Pollywog provides an extension that makes a download button appear below the cell when you export with ``to_lfcalc``. To enable this feature, first run ``%load_ext pollywog.magics`` in a cell, then ``%pollywog autodownload on``. When you export, simply click the button to save the file to your computer.

Step 5: Use in Leapfrog
~~~~~~~~~~~~~~~~~~~~~~~~

1. Open Leapfrog
2. Navigate to your block model, drillhole, or mesh
3. Right-click on "Evaluations" or "Numeric" section
4. Select "Import" → "From File"
5. Choose your ``my_calculations.lfcalc`` file
6. The calculations will appear in your evaluation tree

.. TODO: Add screenshot of Leapfrog import process here
.. .. image:: _static/leapfrog_import_process.png
..    :alt: Importing calculations into Leapfrog
..    :align: center
..    :width: 80%
..
.. |

Common Workflows
----------------

Drillhole Preprocessing
~~~~~~~~~~~~~~~~~~~~~~~

Clean and transform drillhole assay data:

.. code-block:: python

    from pollywog.core import CalcSet, Number

    preprocessing = CalcSet([
        # Remove outliers
        Number("Au_capped", "clamp([Au], 0, 100)",
               comment_equation="Cap gold at 100 g/t"),
        Number("Cu_capped", "clamp([Cu], 0, 5)",
               comment_equation="Cap copper at 5%"),

        # Log transforms for kriging
        Number("Au_log", "log([Au_capped] + 0.01)"),
        Number("Cu_log", "log([Cu_capped] + 0.01)"),
    ])

    preprocessing.to_lfcalc("drillhole_preprocessing.lfcalc")

Block Model Postprocessing
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Process estimated grades in a block model:

.. code-block:: python

    from pollywog.core import CalcSet, Number
    from pollywog.helpers import WeightedAverage

    postprocessing = CalcSet([
        # Back-transform from log space
        Number("Au_est", "exp([Au_log_kriged]) - 0.01"),

        # Apply dilution
        Number("Au_diluted", "[Au_est] * 0.95"),

        # Apply recovery
        Number("Au_recovered", "[Au_diluted] * 0.88"),
    ])

    postprocessing.to_lfcalc("block_postprocessing.lfcalc")

Domain Weighting
~~~~~~~~~~~~~~~~

Combine estimates from multiple domains:

.. code-block:: python

    from pollywog.helpers import WeightedAverage
    from pollywog.core import CalcSet

    domain_weighted = CalcSet([
        WeightedAverage(
            variables=["Au_oxide", "Au_sulfide", "Au_transition"],
            weights=["prop_oxide", "prop_sulfide", "prop_transition"],
            name="Au_composite"
        )
    ])

    domain_weighted.to_lfcalc("domain_weighted.lfcalc")

Working with Helpers
--------------------

Pollywog provides helper functions to simplify common patterns:

.. code-block:: python

    from pollywog.helpers import Sum, Product, Scale, CategoryFromThresholds
    from pollywog.core import CalcSet

    helpers_example = CalcSet([
        # Sum multiple Au assays from different labs/methods
        Sum(["Au_fire_assay", "Au_screen_assay", "Au_leach"], name="Au_total"),

        # Calculate tonnage: volume × density
        Product(["block_volume", "density"], name="tonnage"),

        # Apply dilution factor
        Scale("Au", 0.95, name="Au_diluted"),

        # Categorize by grade thresholds
        CategoryFromThresholds(
            variable="Au",
            thresholds=[0.5, 2.0],
            categories=["low_grade", "medium_grade", "high_grade"],
            name="ore_class"
        ),
    ])

    helpers_example.to_lfcalc("helpers_example.lfcalc")

Visualization in Jupyter
-------------------------

Display calculation sets as interactive HTML in Jupyter notebooks:

.. code-block:: python

    from pollywog.display import display_calcset, display_item, set_theme

    # Set theme (optional)
    set_theme("light")  # or "dark"

    # Display the calcset
    display_calcset(calcset)

    # Display individual items
    display_item(my_number)
    display_item(my_category)

Items and CalcSets also have **automatic rich display** in Jupyter notebooks. Simply evaluating them in a cell will show their rich HTML representation:

.. code-block:: python

    # Automatic rich display (no display_* function needed)
    calcset  # Shows interactive tree view

    my_number  # Shows formatted item with equation

This creates an interactive tree view showing:

- Calculation names
- Expressions
- Dependencies between calculations
- Comments and metadata

.. TODO: Add screenshot of Jupyter notebook display here
.. .. image:: _static/jupyter_display_example.png
..    :alt: Interactive calculation display in Jupyter
..    :align: center
..    :width: 90%
..
.. |

Reading Existing Files
----------------------

Load and modify existing .lfcalc files:

.. code-block:: python

    from pollywog.core import CalcSet

    # Read existing file
    existing = CalcSet.read_lfcalc("existing_calculations.lfcalc")

    # View contents
    print(f"Loaded {len(existing.items)} calculations")
    for item in existing.items:
        print(f"  - {item.name}")

    # Modify
    from pollywog.core import Number
    existing.items.append(
        Number("new_calc", "[existing_var] * 2")
    )

    # Save modified version
    existing.to_lfcalc("modified_calculations.lfcalc")

Tips for Success
----------------

1. **Start Simple**: Begin with basic calculations and add complexity incrementally
2. **Use Comments**: Document your logic with ``comment_equation`` parameter
3. **Test Small**: Export small test files and verify in Leapfrog before scaling up
4. **Check Dependencies**: Use ``item.dependencies`` to see what variables are referenced
5. **Organize Code**: Keep related calculations together in the same CalcSet
6. **Version Control**: Store your Python scripts in Git for traceability
7. **Validate Data**: Use ``clamp()`` to handle invalid inputs (negatives, outliers)
8. **Break It Down**: Split complex expressions into multiple simple calculations

Common Pitfalls
---------------

**Forgetting Square Brackets**

.. code-block:: python

    # Wrong - Au is treated as undefined variable
    Number("result", "Au * 2")

    # Correct - Au is a reference to existing variable
    Number("result", "[Au] * 2")

**Division by Zero**

.. code-block:: python

    # Risky
    Number("ratio", "[a] / [b]")

    # Safe
    Number("ratio", "[a] / ([b] + 1e-10)")
    Number("ratio", "[a] / clamp([b], 0.001)")

**Log of Zero**

.. code-block:: python

    # Risky
    Number("Au_log", "log([Au])")

    # Safe
    Number("Au_log", "log([Au] + 1e-6)")

**Missing Parentheses**

.. code-block:: python

    # Ambiguous - may not compute as intended
    Number("result", "[a] + [b] * [c]")

    # Clear
    Number("result", "[a] + ([b] * [c])")

Next Steps
----------

Now that you understand the basics, explore:

- :doc:`examples` - Complete collection of example notebooks
- :doc:`tutorials` - Step-by-step workflow examples
- :doc:`expression_syntax` - Complete expression syntax reference
- :doc:`helpers_guide` - Detailed helper function documentation
- :doc:`workflow_patterns` - Common patterns and best practices
- :doc:`best_practices` - Production workflow recommendations
- :doc:`api_reference` - Full API documentation

Additional Resources
--------------------

- **GitHub Repository**: https://github.com/endarthur/pollywog
- **JupyterLite Demo**: https://endarthur.github.io/pollyweb (try pollywog in your browser!)
- **Example Notebooks**: See :doc:`examples` for all available notebooks
- **Issue Tracker**: Report bugs or request features on GitHub

Getting Help
------------

If you encounter issues:

1. Check the documentation for similar examples
2. Review the example notebooks in the repository
3. Search existing GitHub issues
4. Open a new issue with a minimal reproducible example