Track notebooks & scripts¶

In addition to tracking Python scripts, LaminDB tracks interactive analyses performed in notebooks.

By calling track() in a notebook or script, input data, and output data get automatically registered associated with the run.

Note

Provenance tracking of notebooks & scripts is analogous to tracking pipelines, scripts & UI data manipulation, see Project flow.

Setup¶

!lamin init --storage ./test-track

import lamindb as ln

ln.settings.verbosity = "hint"

💡 connected lamindb: testuser1/test-track

Initiate tracking¶

Call track() to auto-generate IDs to track data lineage. Copy these into your cell above track().

ln.settings.transform.stem_uid = "9priar0hoE5u"
ln.settings.transform.version = "0"
ln.track()

💡 notebook imports: lamindb==0.72.1

💡 saved: Transform(uid='9priar0hoE5u6K79', version='0', name='Track notebooks & scripts', key='track', type='notebook', created_by_id=1, updated_at='2024-05-25 15:25:57 UTC')

💡 saved: Run(uid='UwHReZ61Qi7MGLa36Y2x', transform_id=1, created_by_id=1)

💡 tracked pip freeze > /home/runner/.cache/lamindb/run_env_pip_UwHReZ61Qi7MGLa36Y2x.txt

Run(uid='UwHReZ61Qi7MGLa36Y2x', started_at='2024-05-25 15:25:57 UTC', is_consecutive=True, transform_id=1, created_by_id=1)

LaminDB now automatically tracks all input and output data.

Save run reports and source artifact¶

If you want to save a notebook including its run report & source artifact, run:

ln.finish()

See how transforms with execution reports looks in LaminHub:

Query for a notebook or script¶

In the API, filter the Transform registry to obtain a notebook record:

import lamindb as ln


transform = ln.Transform.filter(name="Track notebooks").one()
# Your notebook is linked with to its source artifact (stripped of its output cells) and execution report (with the notebook's output cells)
transform.source_artifact
transform.latest_report

On LaminHub, use the UI filter in the Transforms view.