Skip to content

FIX: Prune emptied atomic attacks on scenario resume#2210

Open
romanlutz wants to merge 1 commit into
microsoft:mainfrom
romanlutz:romanlutz-scenario-resume-prune
Open

FIX: Prune emptied atomic attacks on scenario resume#2210
romanlutz wants to merge 1 commit into
microsoft:mainfrom
romanlutz:romanlutz-scenario-resume-prune

Conversation

@romanlutz

Copy link
Copy Markdown
Contributor

Description

On resume, Scenario._apply_persisted_objectives restricts each atomic attack's seed groups to the persisted objective subset via keep_seed_groups_with_hashes. For scenarios that build one atomic attack per objective (for example AdaptiveScenario / TextAdaptive, where every seed group becomes its own AtomicAttack), any atomic attack whose single objective falls outside the persisted subset gets pruned down to zero seed groups. Before this fix those emptied atomic attacks stayed in self._atomic_attacks and would be scheduled with nothing to run.

This change collects only the atomic attacks that still have seed groups after pruning into a retained_attacks list and reassigns self._atomic_attacks, so emptied attacks no longer linger. Bulk scenarios (a single atomic attack holding all objectives) are unaffected, since their one attack never empties as long as at least one persisted objective survives, which is why the bug stayed latent until the per-objective (adaptive) shape hit resume.

Tests and Documentation

  • Added test_resume_discards_per_objective_attacks_outside_persisted_subset in tests/unit/scenario/core/test_scenario.py. It uses a per-objective scenario stand-in to assert that emptied atomics are dropped on resume and that every retained atomic still carries its seed groups.
  • tests/unit/scenario/core/test_scenario.py passes in full (57 passed).
  • JupyText: attempted doc/code/scenarios/3_adaptive_scenarios.py. It currently fails at cell In[3] due to a pre-existing main regression that is unrelated to this change: the flip technique sets a system-prompt seed at sequence 0 which collides with the objective's user prompt at sequence 0, raising ValidationError: Inconsistent roles found for sequence 0: {'system', 'user'}. That failure is on the fresh-run build path, not the resume-prune path touched here.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants