feat(tinker): add TinkerNativeBackend #532
Merged
+3,074
−35
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 5 commits
January 27, 2026 13:32
Separate native Tinker training/inference from LocalBackend to keep the API clear while enabling explicit loss/checkpoint behavior and config.
Align tinker native types with OpenAI tooling and update tests to avoid invalid type expressions under pyright.
Use merge_state for backend persistence to avoid clobbering model state, and fail fast on trajectories without Choice objects to prevent no-op training. Expose policy version fields on trajectories for off-policy tracking.
Add a new PipelineTrainer module that implements an asynchronous 3-stage pipeline (rollout, training, eval) for efficient RL training: - PipelineTrainer: Main trainer class with configurable workers, batch sizes, and off-policy limits - StatusReporter: Live progress reporting with tqdm and periodic logging - PipelineState: Shared state dataclass for stage coordination - Type definitions for RolloutFn, SingleRolloutFn, EvalFn Key features: - Async rollout workers with policy version tracking - Stale sample detection and automatic discard - Zero-variance group handling with collapse detection - Graceful signal handling (SIGINT/SIGTERM) - State persistence for training resumption - Eval scheduling with configurable intervals Also includes: - yes_no_maybe_pipeline.py: Simple example showing basic usage - binary_prefix_tool_pipeline.py: Complex example with tool calls Updates to tinker_native backend: - Add debug logging via ART_TINKER_TRAIN_LOG/ART_TINKER_SAMPLE_LOG - Add fallback for create_conversation_prefix_with_tools - Fix tool_call id handling in OpenAI server responses
- Fix import path for get_free_port (moved from service to server) - Add cast for merge_state return type - Fix test to use async function for TrajectoryGroup creation - Move tinker deps to separate dependency group - Add tinker to allowed-unresolved-imports for ty
corbt
force-pushed
the
tinker-native-backend
branch
from
793b1e0 to
701e54a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
TinkerNativeBackendthat implements native Tinker loss/checkpoint flow with renderer-based data conversion and an in-process OpenAI-compatible server.tinkerdependency group (fastapi/uvicorn/tinker/tinker-cookbook) and conditionally export Tinker backends to keep base installs light.LocalBackendmodelbase_pathwith backend path and add an integration test for the Tinker native flow.This is an alternative to #523. That PR tried to combine native Tinker behavior with ART-native behavior in a single backend, which made the code too complex to reason about. Here we split it into a separate native backend. Downside: args are less compatible with
LocalBackend. Upside: we can take advantage of Tinker's functionality more explicitly.Experimental: not yet tested with multi-turn rollouts or tool-calls, but the yes-no-maybe flow converges (avg reward ~0.955 by step 4).
Test plan
uv run pytest tests/integration/test_tinker_native_backend.py -v -s