Skip to content

fix(manifest): attach ModelHarnessManifest on every mode's result#50

Merged
ernestprovo23 merged 1 commit into
mainfrom
fix/manifest-on-every-result
Jul 14, 2026
Merged

fix(manifest): attach ModelHarnessManifest on every mode's result#50
ernestprovo23 merged 1 commit into
mainfrom
fix/manifest-on-every-result

Conversation

@ernestprovo23

Copy link
Copy Markdown
Member

Summary

The auditable ModelHarnessManifest is documented as first-class on every CouncilResult ("rides on every result, not behind a debug flag"), but in practice only the ask/synthesize path built one. debate, adversarial, and vote constructed their result directly in modes.py and returned with manifest = None, so the invariant silently drifted for 3 of 5 modes and no test guarded it.

The fix — a single-site invariant

All five modes funnel through the one chokepoint Council._cached_run. This PR adds Council._ensure_manifest, called there on every returned result:

  • No-op for the synthesize/raw path (it already builds a richer manifest in _ask_uncached, so it can populate verdict-provenance slots and re-stamp secret-safety).
  • Fills the manifest for debate/adversarial/vote from the resolved membership + collected answers.
  • Covers the zero-members early return and the cache-hit path, so no result can escape without a secret_safety = verified_no_secrets manifest.

Membership is re-resolved via the existing _available_members() (the same resolution _cache_key already performs) rather than threaded through every mode's return — so providers_called/model_ids reflect the full membership even for debate rounds where a member later dropped out, while per-answer receipts come from result.answers.

Verdict scope (deliberate)

Manifest everywhere; verdict scope unchanged. The clustering verdict still runs on synthesize/ask only. It is not forced onto adversarial (which already emits a judge verdict, so it would double-count the same adjudication) and is deferred on debate/vote. Rationale recorded in PDD §4a.

Tests

New tests/test_manifest_all_modes.py pins the invariant per mode: manifest is not None and secret_safety == verified_no_secrets for debate, adversarial, and vote, including the zero-members and cache-hit paths. Full suite: 499 passed, coverage 92.3% (gate 75%), ruff check + ruff format --check clean.

Doc reconciliation

  • Manifest-on-every-result claim is now true — strengthened to state the one-chokepoint invariant across all five modes (README, SYSTEM_CONTEXT_DIAGRAM.md, PDD §4a).
  • The constrained-choice vote mode is documented as shipped (CAC-09 / vote mode — structured majority with reported split #3), not "absorbed by provider_votes" — the two are complementary (a fixed ballot vs. clustered free-form stances). Updated across README, SYSTEM_CONTEXT_DIAGRAM.md, PDD (§1/§3/§4a/§8/§9/§12), DOCUMENTATION_INDEX.md, and CHANGELOG.md. All core docs remain under 500 lines.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QofWZ4MivqEKPevjSYuwdk

The auditable ModelHarnessManifest was documented as first-class on every
CouncilResult ("rides on every result, not behind a debug flag"), but only the
ask/synthesize path built one. debate, adversarial, and vote constructed their
result directly in modes.py and returned with manifest=None, so the invariant
silently drifted for 3 of 5 modes and no test guarded it.

Make manifest attachment a true invariant with a single-site fix: all five
modes funnel through Council._cached_run, which now calls a new
Council._ensure_manifest on every returned result. It is a no-op for the
synthesize/raw path (which builds its own richer manifest in _ask_uncached so
it can populate verdict-provenance slots and re-stamp secret-safety), and fills
the manifest for debate/adversarial/vote from the resolved membership +
collected answers. This also covers the zero-members early return and the
cache-hit path, so no result can escape without a secret-safety-VERIFIED
manifest.

Verdict scope is deliberately unchanged: the clustering verdict still runs on
synthesize/ask only. It is not forced onto adversarial (which already emits a
judge verdict, so it would double-count the same adjudication) and is deferred
on debate/vote; rationale recorded in PDD 4a.

Add regression tests (tests/test_manifest_all_modes.py) asserting
manifest is not None and secret_safety == verified_no_secrets for debate,
adversarial, and vote, including the zero-members and cache-hit paths.

Reconcile docs so (a) the manifest-on-every-result claim is now true and (b)
the constrained-choice vote mode is documented as shipped (CAC-09 / #3), not
"absorbed by provider_votes": README, SYSTEM_CONTEXT_DIAGRAM, PDD,
DOCUMENTATION_INDEX, and CHANGELOG.
@ernestprovo23
ernestprovo23 merged commit 4753b85 into main Jul 14, 2026
6 checks passed
@ernestprovo23
ernestprovo23 deleted the fix/manifest-on-every-result branch July 14, 2026 00:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant