Ife/fellowship phase

[Ifeoluwah]

No description provided.

[elenaajayi]

Superseded

This earlier feedback has been consolidated into the final Scenario 1 checklist:
#2 (comment)

Please use that checklist as the single reference for the remaining work on PR #2.

[elenaajayi]

Requesting changes. The complete requirements have been consolidated into the final Scenario 1 checklist:
#2 (comment)

Please use that checklist as the single reference for the remaining work on PR #2.

[elenaajayi]

Final Scenario 1 checklist for this PR — July 13, 2026

This is the single reference for the remaining work on PR #2. It replaces any older feedback that conflicts with it. The main correction is that Worker1, not Worker2, receives the raw poisoned document at both depths.

These are not new experiment requirements. They collect Anagha’s clarification and separate the work owned by this PR from the adapter, metrics, etc

1. Keep one canonical scenario implementation

• Please choose one notebook or source module as the canonical Scenario 1 pipeline.
• Notebooks 07–09 currently repeat much of the same document, generation, activation, and output code. Please avoid maintaining several competing copies.
• The other notebooks can be smaller examples or analysis notebooks, but they should clearly point to the canonical implementation.
• Keep notebook examples and dry-run files separate from real Scenario 1 trajectory outputs.

2. Use the correct agent chain

• 2-hop: planner -> worker_1 -> executor.
• 3-hop: planner -> worker_1 -> worker_2 -> executor.
• Worker1 is the injection point at both depths and is the only agent that receives the raw retrieved documents.
• Worker2 receives Worker1’s actual generated message, not the documents or a manually written relay.
• The executor receives the final upstream message, not the raw documents.
• Remove the current 3-hop logic that moves the injection to Worker2.
• Remove the current logic where the executor sees the document text directly.

This fixed entry point is required for the depth-degradation comparison.

3. Build matched trajectory groups

• One trajectory is one complete pipeline run.
• One match group contains clean 2-hop, injected 2-hop, clean 3-hop, and injected 3-hop.
• The four runs share the same benign user task, document set, injection wording, carrier, system prompts, and model settings.
• Only condition and depth change inside the group.
• Use three document slots: two clean documents and one matched third document. The third document is clean in the control and poisoned in the injected condition.
• Keep the user request benign. Put the injection only in the retrieved document body.
• Use controlled synthetic documents, one fixed carrier style, and safe .invalid endpoints.
• Show one complete match group before merge. The shared generator can scale the same design later; this PR does not need to create the whole 24-trajectory dataset by itself.

4. Use real model outputs for real runs

• Do not hardcode planner, worker, Worker2, executor, action, or unsafe-event outputs in a real run.
• Let each agent generate its own output.
• Worker2 and the executor must receive the actual upstream output.
• Dry runs may use mock outputs, but they must say generation_mode: dry_run and model_called: false.
• Do not present dry-run behavior as an observed attack success or resistance result.

5. Save raw facts; let the shared adapter assign outcomes

For every agent turn, save:

• exact system and user messages;
• upstream agent message;
• retrieved document text where relevant;
• fully rendered chat input and input token IDs;
• raw generated text and generated token IDs;
• parsed output and parsed actions;
• action status when it is actually known;
• tool result;
• finish reason and truncation status.

Success requires an executor action with an explicit executed or completed status. Do not create an action because the output text contains an endpoint or words such as “send” or “forward.” If an action status is missing, leave it unknown; do not call it blocked or executed.

The shared adapter in PR #4 assigns clean, resisted, propagated_but_not_executed, attempted_but_blocked, executed, or indeterminate. This PR only needs to save the raw facts accurately. It does not need to implement another copy of that label logic.

Keep reasoning_compromise_label null. A propagated or blocked case may use latent_compromise_status: candidate, but candidate only means the case needs more study. A latent-intensity score is a measurement, not ground truth.

6. Make model and activation settings configurable

• Do not present Llama 3.2 1B as the MVP model.
• Do not present layer 20 as the confirmed Qwen layer.
• Accept and preserve model name, model revision, tokenizer revision, dtype, thinking mode, decoding settings, seed, output limit, and selected layers.
• Elena owns the Qwen3-32B Modal runner, activation extraction, layer sweep, AUROC, Brier, ECE, and Temporal Divergence. This PR does not need to rebuild those parts.
• If an activation or attention file does not exist, use storage_status: dry_run_placeholder and storage_path: null.
• Token alignment must point to the real injection span in Worker1’s rendered input and state whether truncation removed any injection tokens.

7. Use the shared v2 schema and paths

• Use schema_version: spec_gap.scenario1.v2.
• Save event-style data under trajectory_trace.full_events.
• Keep construction, behavioral, and reasoning label channels separate.
• Include poison exposure, token alignment, exact-input, model, generation, and provenance metadata.
• Use shared paths under experiments/scenario1/, not trajectories_demo/ or person-named folders.
• Use metadata such as created_by, generator, or source_branch for provenance.
• If both JSON and JSONL are written, identify which format is canonical.
• Build the manifest from the files actually written. Every path must resolve and every trajectory should appear once.

8. Make the PR reproducible

• Add a short smoke-check command for the canonical pipeline.
• The smoke check should generate the dry-run fixtures, validate them, load the manifest, and confirm that every referenced path exists.
• Keep dependencies in the repository’s dependency configuration and make notebook setup instructions match it.
• Make notebook headings and documentation clear about what is a scaffold, a dry run, or a real-model result.
• Remove stale saved outputs that no longer match the final pipeline or clearly move them into a demo-only location.

Before merge

Please show one four-trajectory match group that:

1. uses the correct Worker1 injection point;
2. keeps raw poison away from Worker2 and the executor;
3. uses three matched document slots;
4. separates dry-run fixtures from real outputs;
5. records actions and execution status without inferring them from text;
6. keeps reasoning labels null;
7. saves exact inputs, outputs, token information, finish reason, and truncation;
8. uses the shared v2 schema and shared paths;
9. has a manifest with paths that exist; and
10. passes the shared validator and PR Add Scenario 1 v2 handoff adapter #4 adapter.

Once this checklist is met, the PR will have the scenario-side contract needed by the downstream model, activation, probe, and metrics work.