feat: Switchyard token-capture integration for OpenHands SWE runs (zero-fork)

[linj-glitch]

What

Opt-in routing of OpenHands policy calls through a token-capture Switchyard proxy, so every call's exact vLLM prompt token ids, generation token ids, and sampled-token log probabilities are captured on the wire and returned in Gym's existing trainer-facing Responses API rollout (TokenIDLogProbMixin items). Reward, patch production, and evaluation are unchanged.

Gym SWEBenchWrapper → OpenHands agent (Apptainer) → Switchyard → vLLM
                                        capture records ↘  sessions/<uuid>/*.json
Gym run() ← GET /v1/sessions/<uuid>/completions ← Switchyard

Zero-fork architecture

No nv-OpenHands change; the pinned fork commit and YAML pins do not move. The pinned client is steered entirely by configuration Gym writes per run:

• the agent TOML model becomes the Switchyard route id (Switchyard dispatches on it and substitutes the target model upstream);
• the capture session rides on completion_kwargs.proxy_x_session_id, which lands verbatim in the client's request body — Switchyard resolves it (header > body > launcher API key) and strips it before forwarding;
• the model-server entry of the agent container's NEMO_GYM_CONFIG_DICT is rewritten to Switchyard's host:port (eval container keeps the original).

The body-field session channel is the cross-repo counterpart, landed on Switchyard PR #63 as eb351a7. Gym has no dependency on the Switchyard package — the coupling is a wire contract (OpenAI chat completions in, proxy_x_session_id correlation, versioned record JSON out), guarded by a schema_version gate.

Activation and safety

• Off by default: without switchyard_base_url, generated scripts/configs are byte-identical to today (tested).
• One YAML key to enable (commented examples added to the three OpenHands configs):

  switchyard_base_url: http://<switchyard-host>:<port>   # validated at startup

• Fail-closed for training, fail-open for diagnostics: any retrieval/schema/history failure masks the sample (mask_sample=true, switchyard_trace_error) while preserving the OpenHands patch, eval metrics, and reward. Strict-history reconstruction never repairs a divergent chain.

Changes

• nemo_gym/switchyard_trace.py (new): fail-closed session validation, strict-history reconstruction, tool mapping. Shared infrastructure — depends only on Switchyard's record schema and the shared Responses converter, so any agent server (not just swe_agents) can adopt Switchyard trace collection with it.
• responses_api_agents/swe_agents/app.py: switchyard_base_url config + startup URL validation, per-run session UUID, agent-container TOML/config-dict routing, post-run retrieval + masking in run(), switchyard_trace_error on the verify response.
• gym_switchyard_integration.md: full design doc (updated to the zero-fork architecture).
• Tests: tests/unit_tests/test_switchyard_trace.py (validation matrix, two-call tool-trajectory reconstruction) and test_app.py additions (gating, TOML/config-dict routing, run success/failure/masking, URL validation).

Testing

• swe_agents suite + tests/unit_tests/test_switchyard_trace.py + tests/unit_tests/test_responses_converter.py: 226 passed; ruff clean.
• Switchyard capture suite on the PR Request lifecycle docs #63 branch: 54 passed; full unit suite 2033 passed.
• Cross-repo harness: a two-call tool session shaped exactly like the pinned client's POSTs, driven through Switchyard's real capture processors and retrieval endpoint, reconstructs through this PR's code with exact token triples on the right generated items.
• Remaining: GPU/cluster end-to-end (design doc test Dapo17k #6).

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[linj-glitch]

How to activate Switchyard trace collection (YAML)

Everything is off by default — enabling it is one key on the swe_agents server config. The three OpenHands configs now carry a commented example; uncomment and point it at your proxy:

swe_agents:
  responses_api_agents:
    swe_agents:
      agent_framework: openhands
      # Route policy calls through Switchyard and rebuild the rollout with
      # exact token ids. Must be exactly http://<host>:<port> (validated at
      # startup) and reachable from this process AND the agent container.
      switchyard_base_url: http://switchyard-host:4000

Equivalently, keep the shipped configs untouched and set it per run via a Hydra override / overlay config on top of swebench_openhands.yaml.

Prerequisite — the Switchyard side must be a capture-enabled serve (PR NVIDIA-NeMo/Switchyard#63 at eb351a7+), with a route whose id matches this config's model_server.name (policy_model in the shipped configs):

# single-vllm.yaml
routes:
  policy_model:
    type: model
    target:
      model: <actual-model-served-by-vllm>
      base_url: http://<vllm-host>:<port>/v1
      api_key: dummy
      format: openai
      token_capture_engine: vllm

switchyard \
  --routing-profiles single-vllm.yaml \
  --enable-rl-logging \
  --rl-log-dir /data/switchyard-traces/<job-id> \
  -- serve

(Use the --routing-profiles ... -- serve form: serve --config is the Rust profile server and rejects --enable-rl-logging.)

What you get / what can happen:

key absent	behavior byte-identical to main — no Switchyard codepath runs
enabled, capture healthy	rollout input/output items carry exact prompt_token_ids / generation_token_ids / generation_log_probs; response metadata records the session id + record UUIDs
enabled, anything fails	sample is masked (mask_sample: true, reason in switchyard_trace_error) — patch, eval metrics, and reward are preserved

Full details in gym_switchyard_integration.md.

🤖 Generated with Claude Code