Skip to content

feat(python): rollout store bindings (local + remote)#128

Merged
beinan merged 2 commits into
lance-format:mainfrom
beinan:rollout-python-bindings
Jul 10, 2026
Merged

feat(python): rollout store bindings (local + remote)#128
beinan merged 2 commits into
lance-format:mainfrom
beinan:rollout-python-bindings

Conversation

@beinan

@beinan beinan commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

The rollout store shipped in 0.6.0 with Rust core / client / server support but no Python bindings — Python users couldn't append or read rollouts at all, local or remote. This adds both, mirroring the existing Context / RemoteContext pattern.

  • RolloutStore (sync): embedded via open(uri), or remote via connect(base_url, name) / connect_or_create(...).
  • RemoteRolloutStore (async): awaitable HTTP wrapper (blocking calls run in an executor), the path RL generation workers + the learner use.

Both dispatch through the unified lance-context RolloutStore enum, so a single pyclass covers embedded and remote.

Ergonomics

Rollout rows carry ~35 fields, so records cross the FFI boundary as JSON (serde on the Rust side) rather than 35 keyword args. The Python wrapper accepts:

store.add({"id": "r0", "rollout_id": "traj-1", "reward": 1.0})   # one dict
store.add([{...}, {...}])                                          # many
store.add_one(id="r0", rollout_id="traj-1", reward=1.0)           # kwargs

binary_payload is accepted as raw bytes and base64-encoded to match the DTO's JSON contract; get/list project it out (cheap scans) and get_blob(id) materializes it on demand.

Methods: open / connect / connect_or_create, add, add_one, list, get, get_blob, version, checkout. Exported from lance_context.

Changes

  • python/src/lib.rs: RolloutStore pyclass wrapping the unified enum; JSON marshalling of records/DTOs; registered in the module.
  • python/python/lance_context/api.py: RolloutStore (sync) + RemoteRolloutStore (async) wrapper classes.
  • python/python/lance_context/__init__.py: exports.
  • crates/lance-context/src/lib.rs: re-export CreateRolloutStoreRequest (needed for connect_or_create).
  • python/Cargo.toml: depend on lance-context with the remote feature.

Test plan

  • test_rollout.py — embedded: dict/list/kwargs add, list/get, blob roundtrip (projected out, fetched on demand), reopen sees prior rows, empty-add rejected
  • test_rollout_remote.py — spawns a real lance-context-server subprocess, async connect_or_create + full roundtrip + cross-connection read-your-writes; skips cleanly if the server binary isn't built
  • ruff check + ruff format --check + pyright clean; 7/7 tests pass

🤖 Generated with Claude Code

Beinan Wang and others added 2 commits July 10, 2026 17:56
The rollout store had no Python surface — only Rust core/client/server. This
adds `RolloutStore` (sync, embedded via open() or remote via connect()) and
`RemoteRolloutStore` (async HTTP wrapper), mirroring the existing Context /
RemoteContext bindings.

Both dispatch through the unified `lance-context` RolloutStore enum, so one
pyclass covers embedded and remote. Rollout rows carry ~35 fields, so records
cross FFI as JSON (serde on the Rust side), with `binary_payload` base64 per
the DTO contract; the Python wrapper accepts raw bytes and dict/list[dict] as
well as add_one(**kwargs).

Methods: open / connect / connect_or_create, add, add_one, list, get,
get_blob, version, checkout. Exported from lance_context.

Tests: test_rollout.py (embedded add/list/get/get_blob/reopen, dict+list+
kwargs, blob roundtrip) and test_rollout_remote.py (spawns a real
lance-context-server subprocess, async connect_or_create + roundtrip +
cross-connection read-your-writes; skips if the server binary isn't built).
ruff + pyright clean.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
- Mint a UUID4 for a rollout record when `id` is omitted/empty, so callers
  never have to hand-roll ids (applies to both add and add_one, local + remote).
- Add `artifact_type` as a first-class nullable Utf8 column, orthogonal to
  `content_type`: content_type is the transport/media type (e.g. image/png),
  artifact_type is the user-defined semantic category (e.g.
  excel_grade_screenshot). A real column so it can be filtered / grouped-by /
  projected without materializing the free-form metadata JSON.

Wired through RolloutRecord, Arrow schema (read+write), the API DTOs
(AddRolloutRequest, RolloutRecordDto), the server routes, and the Python
bindings (which pass dicts through generically, so no field allowlist change).

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
@beinan beinan merged commit 12ed99a into lance-format:main Jul 10, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant