feat(python): rollout store bindings (local + remote)#128
Merged
Conversation
The rollout store had no Python surface — only Rust core/client/server. This adds `RolloutStore` (sync, embedded via open() or remote via connect()) and `RemoteRolloutStore` (async HTTP wrapper), mirroring the existing Context / RemoteContext bindings. Both dispatch through the unified `lance-context` RolloutStore enum, so one pyclass covers embedded and remote. Rollout rows carry ~35 fields, so records cross FFI as JSON (serde on the Rust side), with `binary_payload` base64 per the DTO contract; the Python wrapper accepts raw bytes and dict/list[dict] as well as add_one(**kwargs). Methods: open / connect / connect_or_create, add, add_one, list, get, get_blob, version, checkout. Exported from lance_context. Tests: test_rollout.py (embedded add/list/get/get_blob/reopen, dict+list+ kwargs, blob roundtrip) and test_rollout_remote.py (spawns a real lance-context-server subprocess, async connect_or_create + roundtrip + cross-connection read-your-writes; skips if the server binary isn't built). ruff + pyright clean. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
- Mint a UUID4 for a rollout record when `id` is omitted/empty, so callers never have to hand-roll ids (applies to both add and add_one, local + remote). - Add `artifact_type` as a first-class nullable Utf8 column, orthogonal to `content_type`: content_type is the transport/media type (e.g. image/png), artifact_type is the user-defined semantic category (e.g. excel_grade_screenshot). A real column so it can be filtered / grouped-by / projected without materializing the free-form metadata JSON. Wired through RolloutRecord, Arrow schema (read+write), the API DTOs (AddRolloutRequest, RolloutRecordDto), the server routes, and the Python bindings (which pass dicts through generically, so no field allowlist change). Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The rollout store shipped in 0.6.0 with Rust core / client / server support but no Python bindings — Python users couldn't append or read rollouts at all, local or remote. This adds both, mirroring the existing
Context/RemoteContextpattern.RolloutStore(sync): embedded viaopen(uri), or remote viaconnect(base_url, name)/connect_or_create(...).RemoteRolloutStore(async): awaitable HTTP wrapper (blocking calls run in an executor), the path RL generation workers + the learner use.Both dispatch through the unified
lance-contextRolloutStoreenum, so a single pyclass covers embedded and remote.Ergonomics
Rollout rows carry ~35 fields, so records cross the FFI boundary as JSON (serde on the Rust side) rather than 35 keyword args. The Python wrapper accepts:
binary_payloadis accepted as rawbytesand base64-encoded to match the DTO's JSON contract;get/listproject it out (cheap scans) andget_blob(id)materializes it on demand.Methods:
open/connect/connect_or_create,add,add_one,list,get,get_blob,version,checkout. Exported fromlance_context.Changes
python/src/lib.rs:RolloutStorepyclass wrapping the unified enum; JSON marshalling of records/DTOs; registered in the module.python/python/lance_context/api.py:RolloutStore(sync) +RemoteRolloutStore(async) wrapper classes.python/python/lance_context/__init__.py: exports.crates/lance-context/src/lib.rs: re-exportCreateRolloutStoreRequest(needed forconnect_or_create).python/Cargo.toml: depend onlance-contextwith theremotefeature.Test plan
test_rollout.py— embedded: dict/list/kwargs add, list/get, blob roundtrip (projected out, fetched on demand), reopen sees prior rows, empty-add rejectedtest_rollout_remote.py— spawns a reallance-context-serversubprocess, asyncconnect_or_create+ full roundtrip + cross-connection read-your-writes; skips cleanly if the server binary isn't builtruff check+ruff format --check+pyrightclean; 7/7 tests pass🤖 Generated with Claude Code