Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 31 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,33 @@ All notable changes to conclave are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Fixed

- **`ModelHarnessManifest` now rides on *every* mode's result — a true invariant.**
The auditable manifest was documented as first-class on every `CouncilResult`, but
`debate`, `adversarial`, and `vote` built their result directly in `modes.py` and
returned with `manifest = None` (only `ask`/synthesize attached one). The fix moves
manifest attachment to the single chokepoint every mode funnels through
(`Council._cached_run` → new `Council._ensure_manifest`): it fills the manifest from
the resolved membership + collected answers for `debate`/`adversarial`/`vote`
(including the zero-members early return and cache hits), is a no-op for the
synthesize/raw path that already builds its own richer manifest, and stamps
`secret_safety = verified_no_secrets` when the manifest is provably clean. Regression
tests (`tests/test_manifest_all_modes.py`) pin the invariant per mode so it cannot
drift again. The clustering **verdict** scope is unchanged: it still runs on
`synthesize`/`ask` only and is intentionally not layered onto `adversarial` (which
already emits a judge verdict) — see PDD §4a.

### Documentation

- Reconciled `README.md`, `SYSTEM_CONTEXT_DIAGRAM.md`, and the PDD so the
manifest-on-every-result claim is now accurate, and documented the constrained-choice
**`vote` mode** as **shipped** (CAC-09 / #3) rather than "absorbed by `provider_votes`"
— the two are complementary (a fixed ballot vs. clustered free-form stances), not the
same feature. Added the `--mode vote --choices` CLI example and a §4a verdict-scope note.

## [1.1.0] - 2026-06-21

The **auditable council**. Every run now produces a structured, agreement-scored,
Expand Down Expand Up @@ -58,10 +85,10 @@ reproducible arithmetic over the model's clustering — never an LLM-emitted fig

### Note

- **`vote` mode (council issue #3) is absorbed/superseded** by the verdict work:
`provider_votes` records which provider took which position (with evidence) and
`consensus_label`/`consensus_score` report the split deterministically — no separate
`vote` mode is shipped or planned.
- **`vote` (council issue #3):** at 1.1.0 the verdict's `provider_votes` +
`consensus_label`/`consensus_score` were considered to subsume it. **Superseded post-1.1.0**
— a real constrained-choice `vote` mode later shipped (CAC-09 / #3; see `[Unreleased]`),
complementary to `provider_votes`: a fixed ballot vs. clustered free-form stances.

## [1.0.0] - 2026-06-14

Expand Down
5 changes: 3 additions & 2 deletions DOCUMENTATION_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ the canonical authority spec on top of those.

| Doc | Path | Purpose |
|-----|------|---------|
| **Product Design Document** | [`docs/PRODUCT_DESIGN_DOCUMENT.md`](docs/PRODUCT_DESIGN_DOCUMENT.md) | **Canonical** product spec and roadmap, reframed in v1.1 to **the auditable multi-model council**. Problem & vision, personas, BYO-keys/key-handling security, council modes (synthesize/raw/debate/adversarial; `vote` absorbed by the verdict), the v1.1 auditable verdict (§4a — CouncilResult v2, deterministic consensus, native structured output, the manifest), provider matrix, architecture, scope + non-goals, demand-gated v1.2 roadmap, the mcp-warden dev-time boundary, licensing & positioning, open questions. **When docs disagree, the PDD wins.** |
| **Product Design Document** | [`docs/PRODUCT_DESIGN_DOCUMENT.md`](docs/PRODUCT_DESIGN_DOCUMENT.md) | **Canonical** product spec and roadmap, reframed in v1.1 to **the auditable multi-model council**. Problem & vision, personas, BYO-keys/key-handling security, council modes (synthesize/raw/debate/adversarial/vote — `vote` shipped as a constrained-choice ballot, CAC-09 / #3, complementary to the verdict's `provider_votes`), the v1.1 auditable verdict (§4a — CouncilResult v2, deterministic consensus, native structured output, the manifest), provider matrix, architecture, scope + non-goals, demand-gated v1.2 roadmap, the mcp-warden dev-time boundary, licensing & positioning, open questions. **When docs disagree, the PDD wins.** |

---

Expand Down Expand Up @@ -81,7 +81,7 @@ Run: `pytest` (config in `pyproject.toml`, `asyncio_mode = "auto"`).
|------|------|---------|
| Packaging | [`pyproject.toml`](pyproject.toml) | hatchling build, deps (httpx, pydantic, rich, typer, pyyaml — no LLM SDK), dev extras, console script, pytest config. License: MIT. **PyPI distribution name `conclave-cli`** (the name `conclave` is an unrelated project); command + import stay `conclave`. |
| Release runbook | [`RELEASING.md`](RELEASING.md) | Operator runbook: one-time PyPI OIDC Trusted-Publisher setup for `conclave-cli`, cut-a-release checklist (bump→tag→publish Release), post-release verification (Sigstore bundle, PEP 740 attestations), rollback/yank. |
| Changelog | [`CHANGELOG.md`](CHANGELOG.md) | Keep-a-Changelog history per release (SemVer). The `[1.1.0]` (unreleased) entry covers the auditable council — CouncilResult v2, deterministic consensus, native structured output, the manifest, and the absorbed `vote` mode; the 1.0.0 entry covers the distribution rename, key-leak hardening, synthesizer versioning, and release engineering. |
| Changelog | [`CHANGELOG.md`](CHANGELOG.md) | Keep-a-Changelog history per release (SemVer). The `[Unreleased]` entry covers the manifest-on-every-result invariant fix and the `vote`-mode-shipped doc reconciliation; `[1.1.0]` covers the auditable council — CouncilResult v2, deterministic consensus, native structured output, the manifest; the 1.0.0 entry covers the distribution rename, key-leak hardening, synthesizer versioning, and release engineering. |
| Dev lockfile | [`requirements-dev.lock`](requirements-dev.lock) | Hash-pinned dev + runtime tree for reproducible installs/CI. Regenerate via `uv pip compile --universal --generate-hashes --python-version 3.11 --extra dev pyproject.toml -o requirements-dev.lock`. |
| License | [`LICENSE`](LICENSE) | MIT License. Copyright (c) 2026 Ernest Provo. Matches the `pyproject.toml` license field. |
| Security policy | [`SECURITY.md`](SECURITY.md) | BYO-keys vulnerability reporting policy: how to report, scope, and the key-handling guarantees consumers can rely on. |
Expand All @@ -100,6 +100,7 @@ Run: `pytest` (config in `pyproject.toml`, `asyncio_mode = "auto"`).

| Date | Change |
|------|--------|
| 2026-07-13 | **Manifest-on-every-result invariant fix + doc reconciliation.** `ModelHarnessManifest` now attaches on all five modes (was `None` for `debate`/`adversarial`/`vote`) via a single-site fix at `Council._cached_run` → new `_ensure_manifest`; regression tests in `tests/test_manifest_all_modes.py`. Docs corrected so the manifest-on-every-result claim is accurate and the `vote` mode is documented as **shipped** (CAC-09 / #3), not "absorbed" — across README, `SYSTEM_CONTEXT_DIAGRAM.md`, PDD (§1/§3/§4a/§8/§9/§12), and `CHANGELOG.md` (`[Unreleased]`). Verdict scope unchanged (still `synthesize`/`ask`-only; not layered on `adversarial`). |
| 2026-06-21 | **v1.1 docs pivot — the auditable multi-model council.** PDD reframed to the auditable-council wedge (new §4a: CouncilResult v2, deterministic `position_cluster_ratio_v1` consensus, native + fallback structured output, the verdict-optional rule, the secret-free `ModelHarnessManifest`); `vote` mode marked **absorbed** by `provider_votes` across PDD §1/§4/§8/§9/§12; demand-gated v1.2 "Operable Council" roadmap. System Context diagram gains the verdict pipeline + manifest (mermaid re-validated). README gains an "Auditable verdict" section (CLI panel + library example). `CHANGELOG.md` `[1.1.0]` added. v0.x mode-detail prose archived to [`docs/archive/pdd-v0.x-modes-detail.md`](docs/archive/pdd-v0.x-modes-detail.md) to keep the PDD under 500 lines. New modules documented: `verdict.py`, `agreement.py`, `verdict_synthesis.py`, `manifest.py`. |
| 2026-06-14 | v1.0.0 release. Version bump 0.3.0 → 1.0.0; new `CHANGELOG.md` (Keep-a-Changelog). Integrates the three v1.0 PRs below. |
| 2026-06-14 | v1.0 distribution + release-engineering (PR-A): PyPI distribution name → `conclave-cli` (command + import stay `conclave`); OIDC Trusted-Publishing release workflow (`.github/workflows/release.yml`, SHA-pinned, PEP 740 attestations, Sigstore keyless signing, inert until a Release fires + publisher configured); `pip-audit` fail-closed CI job in `test.yml`; hash-pinned `requirements-dev.lock`; `RELEASING.md` operator runbook. |
Expand Down
20 changes: 14 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,11 @@ verdict you can act on — structured, scored for agreement, fully auditable**:
`CouncilVerdict` exposing agreement, `conflicts`, `minority_reports`, and `provider_votes`;
a deterministic `consensus_score` (arithmetic over the model's clustering, *never* an
LLM-emitted number); and a redacted `ModelHarnessManifest` recording how the run executed and
which model produced the disagreement analysis. The verdict is **default-on**. (A `vote` mode
was once on the roadmap; it is **absorbed** by `provider_votes` — see below.)
which model produced the disagreement analysis. The verdict is **default-on**, and the manifest
rides on **every** result — for all five modes (`synthesize`, `raw`, `debate`, `adversarial`,
`vote`), not just `ask`. A constrained-choice **`vote` mode** (`--mode vote --choices ...`) also
shipped in v1.1 (CAC-09 / #3) — distinct from the verdict's `provider_votes`, which score
free-form agreement rather than tally a fixed ballot.

See the canonical spec and design docs:

Expand Down Expand Up @@ -118,16 +121,21 @@ conclave ask "Is a service mesh worth it for 8 services?" \
conclave ask "Defend event sourcing for this ledger." \
-c grok,gemini,perplexity --mode adversarial --proposer grok

# Machine-readable output (works for every mode; carries rounds/adversarial too)
# Vote: each member picks one labelled choice; plurality winner (or split) is tallied
conclave ask "Which datastore for this workload?" \
-c grok,gemini,claude --mode vote --choices "Postgres,DynamoDB,MongoDB"

# Machine-readable output (works for every mode; carries rounds/adversarial/vote too)
conclave ask "..." -c grok,perplexity --mode debate --json
```

Mode flags at a glance: `--mode synthesize|raw|debate|adversarial`. `--rounds N`
Mode flags at a glance: `--mode synthesize|raw|debate|adversarial|vote`. `--rounds N`
(default 2) is the *maximum* round count for `debate`; `--converge-threshold FLOAT`
(or `--converge`/`--no-converge`) optionally stops a debate early once answers
stabilize round-over-round (off by default — `--rounds` runs in full). `--proposer
NAME` (default: first member) applies to `adversarial`. `--synthesizer/-s` overrides
the synthesizer *and* the adversarial judge.
NAME` (default: first member) applies to `adversarial`. `--choices "A,B,C"` (two or
more) is required for `vote`. `--synthesizer/-s` overrides the synthesizer *and* the
adversarial judge. Every mode's result carries the auditable `ModelHarnessManifest`.

`--council` accepts either a comma-separated list of friendly names or the name
of a council defined in your config (see below). The built-in `default` council
Expand Down
8 changes: 6 additions & 2 deletions SYSTEM_CONTEXT_DIAGRAM.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ flowchart TB
cli["CLI · conclave ask / providers (cli.py)"]
lib["Library API · from conclave import Council (__init__.py)"]
council["Council orchestrator<br/>fan_out · synthesize_blocks · skip-no-key (council.py)"]
modes["Deliberation modes<br/>debate · adversarial (modes.py + prompts.py)"]
modes["Deliberation modes<br/>debate · adversarial · vote (modes.py + prompts.py)"]
registry["Registry · name to model-id<br/>key PRESENCE only, never values (registry.py)"]
config["Config loader · custom endpoints (config.py)"]
models["Result contract · CouncilResult v2<br/>answers · verdict · consensus · manifest (models.py)"]
Expand Down Expand Up @@ -147,7 +147,11 @@ flowchart TB
with a prompt-level fallback for providers without strict support. The **`ModelHarnessManifest`**
(`manifest.py`) rides on **every** result — first-class, not a debug flag — recording which
model + prompt version produced the clustering (provenance) and stamping `secret_safety`
only after the serialized manifest is scanned clean. A verdict is *optional*: open-ended
only after the serialized manifest is scanned clean. This is enforced as a true invariant at
the single chokepoint `Council._cached_run` → `_ensure_manifest`: **all five modes**
(`synthesize`, `raw`, `debate`, `adversarial`, `vote`) funnel through it, so a result can
never escape without a manifest — including the zero-members early return and cache hits.
A verdict is *optional*: open-ended
prompts, fewer than two responding members, or extraction failure leave `verdict = None`
with the synthesis and member answers intact and the reason recorded on the manifest.
- **Streaming shares the same boundary (PDD §9 #5).** A `--stream` run (and the library
Expand Down
Loading