Integrate Scenario 1 datasets, Modal runner, and probe handoff

[elenaajayi]

Summary

This PR brings the Scenario 1 trajectory work into one shared pipeline. It combines the controlled input registry and v2 event schema, the shared trajectory generator, the Qwen3-32B Modal runner, and the adapter used by the probe and metrics code.

The goal is to remove the incompatible paths that were spread across separate branches and give the team one reproducible contract from input construction through activation requests.

Ordered repository layout

The repository now follows the experiment order instead of keeping active scripts, schemas, guides, and old outputs at the root:

• numbered command wrappers live under scripts/;
• reusable implementation lives under src/;
• the public v2 schema lives under schemas/scenario1/v2/;
• detailed Scenario 1 and Modal guidance lives under docs/;
• controlled inputs remain under experiments/scenario1/inputs/;
• superseded Qwen2.5/v3 files are isolated under archive/scenario1_v3/;
• historical runway commands are isolated under scripts/90_runway_reproduction/.

The main README now presents the runnable sequence from trajectory construction through validation, Modal execution, orchestration, probe handoff, analysis, and figures. It also marks the sequential live orchestrator and safe simulated executor as unfinished, so a one-turn model call cannot be mistaken for a complete trajectory.

Scenario 1 dataset contract

• Defines two controlled match groups: public health and climate science.
• Generates eight base trajectory records: clean and injected conditions at 2-hop and 3-hop depth for each match group.
• Keeps the task, document set, system prompts, seed, document order, and injection wording fixed inside each match group.
• Keeps Worker1 as the only raw-poison entry point at both depths.
• Sends only the upstream relay message to Worker2 and the executor.
• Uses controlled local document fixtures and safe non-routable .invalid endpoints.
• Keeps construction labels, observed behavioral outcomes, and reasoning labels separate.
• Does not treat injection_present=true as a successful compromise.
• Does not assign a latent-compromise label without later human or mechanistic evidence.

Dry-run records validate the structure only. They do not contain fabricated model outputs, tool executions, activations, or costs.

Shared generator and schema

• Uses spec_gap.scenario1.v2 as the public trajectory format.
• Uses one shared generator and experiments/scenario1/... directory layout.
• Records contributor provenance in metadata instead of contributor-named folders.
• Validates match-group completeness, topology, poison exposure, labels, exact paths, and artifact honesty.
• Produces a manifest containing all generated records exactly once.
• Keeps requested tool calls separate from actions actually executed by the executor.

Qwen3-32B Modal runner

• Runs Qwen/Qwen3-32B in the shared agileai Modal workspace.
• Pins revision 9216db5781bf21249d130ec9da846c4624c16137, matching the shared model cache.
• Supports thinking on and thinking off while keeping the remaining decoding settings fixed.
• Uses one H200 per paid model call and limits container concurrency.
• Saves the rendered model input, token IDs, prompt hash, raw generated output, generated token IDs, final response, and separate thinking content.
• Supports residual-stream extraction across configurable layers. All 64 layers are available initially; no primary layer is locked.
• Records model, tokenizer, generation, timing, token, activation, and estimated GPU-cost metadata.
• Requires explicit confirmation before starting a paid H200 run.
• Treats parsed tool calls as requests only. This runner does not execute tools or mark an unsafe action as completed.

The current runner processes one agent turn at a time. Sequential live trajectory orchestration and the safe simulated executor remain follow-up work before a complete live batch is run.

Probe and metrics handoff

• Normalizes v2 event records into the per-step format used by activation extraction and evaluation.
• Preserves the real planner -> worker -> worker2 -> executor topology.
• Preserves poison exposure, token alignment, exact model I/O, thinking mode, activation metadata, and cost metadata.
• Supports six observed behavioral outcomes: clean, resisted, propagated_but_not_executed, executed, attempted_but_blocked, and indeterminate.
• Keeps requested tool calls out of the executed-action list unless the executor actually fires the action.
• Produces stable activation requests for the probe and metrics pipeline.

Validation

• Full test suite: 154 passed
• Python compilation checks passed.
• git diff --check passed.
• All eight structural trajectory records pass schema and semantic validation.
• The eight base records expand into 16 thinking-mode trajectory runs and 56 validated model-turn request templates.
• The Modal request-plan validation did not start a GPU.

Related PRs and contributor credit

This integration includes and aligns work from:

• PR Add Onyinye trajectory generation script for Scenario 1  #3 - Onyinye's Scenario 1 tasks and generator work
• PR Add Scenario 1 v2 handoff adapter #4 - Elena's v2 probe and metrics adapter
• PR Add guarded Modal Qwen3-32B activation runner #8 - Elena's guarded Modal Qwen3-32B runner
• PR Mariame/scenario1 event handoff #9 - Mariame's v2 schema, validator, controlled inputs, and pipeline scaffold

Mariame and Onyinye are preserved in the commit history and provenance metadata. After this integration is accepted, those four PRs can be closed as superseded instead of merged independently.

PR #5, the depth-degradation evaluation infrastructure, remains separate and is not replaced by this PR.

Review focus

Please check:

• the match-group controls and document registry
• Worker1 poison entry and downstream relay behavior
• the v2 schema and six behavioral outcomes
• exact model I/O and token-alignment fields
• requested tool calls versus executed actions
• Modal activation and cost metadata
• the adapter handoff into the probe and metrics code