Dataset Version 1 Still Contains the Deleted Row
A four-row Lance experiment traces append, delete and pinned reads to show what a reproducible training-data reference must include.
original reader, version 1: [1, 2, 3]
reader opened at version 2: [1, 2, 3, 4]
fresh reader after deletion: [1, 3, 4]
explicit version 1: [1, 2, 3]
All four readers point to the same dataset directory. They disagree because they opened different snapshots.
This is the behavior I want when reproducing an old model evaluation, and the behavior I need to remember when removing a bad training example. “The dataset path” does not fully specify either operation.
Retrospective dated August 30; local reproduction on September 12, 2026. I used pylance==10.0.0, released before the article date, with PyArrow 25.0.1 and Python 3.12.8. The corresponding source tag resolves to 95f2f36b22043c3face00afe088c34e0742d01df.
A Dataset Small Enough to Inspect
The fixture contains an integer ID and a three-element float vector. The vectors are synthetic values, not embeddings from a model. They make this a typed columnar dataset without introducing inference, downloads or a nearest-neighbor index.
I wrote IDs 1, 2, 3 to a new temporary directory, retained the returned reader, and appended ID 4 through a second write:
old = lance.write_dataset(initial_table, uri)
appended = lance.write_dataset(extra_table, uri, mode="append")
before_delete = lance.dataset(uri)
appended.delete("id = 2")
latest = lance.dataset(uri)
pinned = lance.dataset(uri, version=1)
The complete executable fixture builds both Arrow tables, verifies their row IDs and prints each reader's version. It uses no shared bucket or existing dataset.
Each operation produced a new visible state. The first write created version 1. Appending produced version 2. Deleting ID 2 produced version 3. The original reader continued to report version 1; the reader opened just before deletion continued to report version 2.
The underlying design explains the result. Lance uses immutable dataset versions, and row deletion can be represented by deletion metadata rather than immediately rewriting the original data files. Its read/write documentation describes the resulting snapshot behavior. The experiment above verifies that behavior for the pinned wheel on local storage; it says nothing about throughput or concurrent object-store writers.
The Path Was the Incomplete Part
Suppose an evaluation log records training.lance, a model checkpoint and a random seed. Later, a cleaning job removes an example. A rerun that opens the path without a version will now see different rows even if the training code and seed are identical.
The storage API exposes the missing information directly. LanceDataset.version returns the checked-out version. checkout_version returns a reader for a requested snapshot while reusing cache state. Neither requires making an old snapshot the latest one. In contrast, restore is documented as creating a new commit that makes the selected contents current. These are distinct operations in the pinned dataset implementation.
| Record in an ML run | Reason |
|---|---|
| Dataset URI and version | Identifies the snapshot that supplied rows |
| Branch identity, when used | Removes ambiguity from branch-local version references |
| Filter and selected columns | Describes which portion of that snapshot was consumed |
| Preprocessing code revision | Identifies transformations applied after reading |
| Sampling and ordering choices | Captures how rows became training batches |
Only the first row was exercised end to end here. The remaining rows are my proposed run-record contract: a versioned store cannot recover a filter or preprocessing choice that the application failed to save.
Retention Is Part of Reproducibility
Version 1 remained readable because the required history and data were still present. That is an operational condition, not a permanent promise attached to the integer 1.
The same Python API provides cleanup_old_versions. Its documentation states that removing old versions and their referenced files can make those snapshots unavailable for checkout or restore. I did not invoke cleanup in this experiment. The old-row result should therefore be read as a snapshot-retention result, not evidence that deleted content can always be recovered.
For an evaluation corpus, I would connect retention policy to the run registry: snapshots referenced by results that still need to be reproduced must remain available. A cleaning job should also record which new version excludes the rejected data, so future runs select that version deliberately.
Four rows were enough to expose the distinction. Removing ID 2 changed the latest dataset correctly. It did not rewrite the input that the original reader had already selected. Recording that reader's version turns an otherwise ambiguous path into a usable part of an experiment record.