Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# =============================================================================
# Release & publish workflow (conclave v1.0 — distribution + release-eng)
# Release & publish workflow (conclave — distribution + release engineering)
#
# !!! INERT UNTIL CONFIGURED — SAFE TO MERGE !!!
# !!! RELEASE-TRIGGERED; FAILS CLOSED ON PUBLISHER MISMATCH !!!
#
# This workflow does NOTHING until BOTH of the following are true:
# (a) a GitHub *Release* is published (a pushed git tag ALONE does not trigger
# it — the trigger is `release: published`, the explicit human gesture), AND
# (b) the `conclave-cli` PyPI project + its OIDC Trusted Publisher are configured
# by the owner (see RELEASING.md "One-time PyPI setup").
# (b) the `conclave-cli` PyPI project trusts this organization repository and
# workflow through OIDC (see RELEASING.md "Trusted Publisher prerequisite").
#
# Until (a) AND (b) hold, merging this file changes nothing at runtime: no tag is
# cut here, no version is bumped here, nothing is published here. Cutting v1.0.0 is
# then a single tag + GitHub Release away (the full runbook is in RELEASING.md).
# cut here, no version is bumped here, and nothing is published here. The full
# operator runbook is in RELEASING.md.
#
# What it does WHEN a Release is published:
# build — builds sdist + wheel with `python -m build`, uploads them as
Expand Down Expand Up @@ -89,7 +89,8 @@ jobs:
# No API token, no stored secret: pypa/gh-action-pypi-publish mints a short-lived
# OIDC token from this job's `id-token: write` and exchanges it for a PyPI upload
# token. PyPI must have a Trusted Publisher (pending or active) for `conclave-cli`
# pointing at owner=ernestprovo23, repo=conclave, workflow=release.yml — see
# pointing at owner=DataScience-EngineeringExperts, repo=conclave,
# workflow=release.yml — see
# RELEASING.md. Until that exists, this job fails CLOSED (publish denied); it
# never falls back to anything insecure.
#
Expand Down Expand Up @@ -154,7 +155,7 @@ jobs:
# Self-check: verify what we just signed against THIS workflow's own
# identity so a broken run never publishes a bad bundle.
verify: true
verify-cert-identity: "https://github.com/ernestprovo23/conclave/.github/workflows/release.yml@${{ github.ref }}"
verify-cert-identity: "https://github.com/DataScience-EngineeringExperts/conclave/.github/workflows/release.yml@${{ github.ref }}"
verify-oidc-issuer: "https://token.actions.githubusercontent.com"
# release-signing-artifacts defaults to true: on a release event the
# .sigstore bundles are attached to the Release assets automatically.
Expand Down
36 changes: 28 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [1.2.0] - 2026-07-18

### Added

- **Elite Decision Protocol (unreleased).** A quality-first `elite` mode for consequential
- **Elite Decision Protocol.** A quality-first `elite` mode for consequential
decisions: independent member answers → concurrent council-wide claim audits → concurrent
member revisions → the existing synthesis and canonical execution-traceable verdict. It is available
through `--mode elite`, `Council.elite()`, and `elite_sync()` in source; no version, tag, or
package publication is claimed here.
through `--mode elite`, `Council.elite()`, and `elite_sync()`.
- **Constrained-choice vote mode.** `--mode vote --choices "A,B,C"`, `Council.vote()`,
and `vote_sync()` collect a fixed labelled ballot and return a plurality winner or split.
This is complementary to verdict `provider_votes`, which cluster free-form stances.
- **Fixed three-success phase gate.** Each of Elite's `initial`, `critique`, and `revision`
phases requires three successful responders. Larger councils may survive partial failures
while three remain. A failed gate stops later calls and returns an incomplete result with a
Expand Down Expand Up @@ -43,6 +47,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- **OpenAI strict verdict extraction.** The extraction schema now requires every declared
object property, excludes engine-computed per-conflict consensus and optional vote confidence,
and represents semantic optionals with honest empty values. Frozen replay fixtures recursively
validate recorded verdict payloads against the exact extraction contract.

- **`ModelHarnessManifest` now rides on *every* mode's result — a true invariant.**
The auditable manifest was documented as first-class on every `CouncilResult`, but
`debate`, `adversarial`, and `vote` built their result directly in `modes.py` and
Expand All @@ -58,10 +67,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`synthesize`/`ask` only and is intentionally not layered onto `adversarial` (which
already emits a judge verdict) — see PDD §4a.

### Changed

- **Organization release identity.** Package project URLs, release signing identity, security
advisory link, and operator runbook now target
`DataScience-EngineeringExperts/conclave`. PyPI Trusted Publishing must trust that exact
organization repository and `release.yml`; publishing continues to fail closed on mismatch.
- **Complete CLI mode help.** `conclave ask --help` now describes all six published modes,
and the module overview includes the `eval` command.

### Documentation

- Added Elite CLI/library examples and reconciled the PDD, system diagram, README, and index
around its unreleased status, fixed gate, partial-failure behavior, phased receipts, and
around its release status, fixed gate, partial-failure behavior, phased receipts, and
latency/cost tradeoff.
- Reconciled `README.md`, `SYSTEM_CONTEXT_DIAGRAM.md`, and the PDD so the
manifest-on-every-result claim is now accurate, and documented the constrained-choice
Expand Down Expand Up @@ -124,7 +142,7 @@ reproducible arithmetic over the model's clustering — never an LLM-emitted fig

- **`vote` (council issue #3):** at 1.1.0 the verdict's `provider_votes` +
`consensus_label`/`consensus_score` were considered to subsume it. **Superseded post-1.1.0**
— a real constrained-choice `vote` mode later shipped (CAC-09 / #3; see `[Unreleased]`),
— a real constrained-choice `vote` mode later shipped (CAC-09 / #3; see `[1.2.0]`),
complementary to `provider_votes`: a fixed ballot vs. clustered free-form stances.

## [1.0.0] - 2026-06-14
Expand Down Expand Up @@ -199,6 +217,8 @@ on top of v0.3.0.
exit-code contract and httpx client lifecycle hardening; transport/CLI/logging
test backfill; first public release with community files.

[1.1.0]: https://github.com/ernestprovo23/conclave/compare/v1.0.0...HEAD
[1.0.0]: https://github.com/ernestprovo23/conclave/compare/v0.3.0...v1.0.0
[0.3.0]: https://github.com/ernestprovo23/conclave/releases/tag/v0.3.0
[Unreleased]: https://github.com/DataScience-EngineeringExperts/conclave/compare/v1.2.0...HEAD
[1.2.0]: https://github.com/DataScience-EngineeringExperts/conclave/compare/v1.1.0...v1.2.0
[1.1.0]: https://github.com/DataScience-EngineeringExperts/conclave/compare/v1.0.0...v1.1.0
[1.0.0]: https://github.com/DataScience-EngineeringExperts/conclave/compare/v0.3.0...v1.0.0
[0.3.0]: https://github.com/DataScience-EngineeringExperts/conclave/releases/tag/v0.3.0
17 changes: 9 additions & 8 deletions DOCUMENTATION_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ context diagram, and this index linking everything together. The Product Design
the canonical authority spec on top of those.

- **Repo:** `/Users/ernestprovo/dev/conclave/`
- **Version:** 1.1.0 · **License:** MIT
- **Version:** 1.2.0 · **License:** MIT
- **Last updated:** 2026-07-18

---
Expand All @@ -15,15 +15,15 @@ the canonical authority spec on top of those.

| # | Doc | Path | Purpose |
|---|-----|------|---------|
| 1 | **README** (project overview) | [`README.md`](README.md) | Install, BYO-keys, five released-mode CLI/library quickstart, source-only Elite, and the gated paid exploratory eval lane. |
| 1 | **README** (project overview) | [`README.md`](README.md) | Install, BYO-keys, six-mode CLI/library quickstart including Elite, and the gated paid exploratory eval lane. |
| 2 | **System Context Diagram** | [`SYSTEM_CONTEXT_DIAGRAM.md`](SYSTEM_CONTEXT_DIAGRAM.md) | Mermaid context for the provider highway, released/source modes, verdict pipeline, eval live guard, manifest receipts, and mcp-warden boundary. |
| 3 | **Documentation Index** | [`DOCUMENTATION_INDEX.md`](DOCUMENTATION_INDEX.md) | This file. Master map of all docs + source layout. |

## Authority spec

| Doc | Path | Purpose |
|-----|------|---------|
| **Product Design Document** | [`docs/PRODUCT_DESIGN_DOCUMENT.md`](docs/PRODUCT_DESIGN_DOCUMENT.md) | **Canonical** product spec: released/source modes, the v1.1 execution-traceable verdict, manifest, paid exploratory evaluation boundary, architecture, and roadmap. **When docs disagree, the PDD wins.** |
| **Product Design Document** | [`docs/PRODUCT_DESIGN_DOCUMENT.md`](docs/PRODUCT_DESIGN_DOCUMENT.md) | **Canonical** product spec: six released modes, the v1.1 execution-traceable verdict, v1.2 Elite and evaluation guard, manifest, architecture, and roadmap. **When docs disagree, the PDD wins.** |

## Product plans

Expand All @@ -46,9 +46,9 @@ Package root: `src/conclave/` (installed as the `conclave` package; console scri

| Module | Path | Responsibility |
|--------|------|----------------|
| Package API | [`src/conclave/__init__.py`](src/conclave/__init__.py) | Public exports include `Council`, result types, verdict/manifest surface, and unreleased `EliteResult` plus Elite protocol constants. |
| Council | [`src/conclave/council.py`](src/conclave/council.py) | Five released async/sync modes plus source-only Elite; Elite keeps protocol completion separate from `ready`/`not_ready`/`indeterminate` decision readiness. |
| Modes | [`src/conclave/modes.py`](src/conclave/modes.py) | Debate, adversarial, vote, and unreleased Elite orchestration; Elite gates initial/claim-audit/revision at three successes. |
| Package API | [`src/conclave/__init__.py`](src/conclave/__init__.py) | Public exports include `Council`, result types, verdict/manifest surface, `EliteResult`, and Elite protocol constants. |
| Council | [`src/conclave/council.py`](src/conclave/council.py) | Six released async/sync modes; Elite keeps protocol completion separate from `ready`/`not_ready`/`indeterminate` decision readiness. |
| Modes | [`src/conclave/modes.py`](src/conclave/modes.py) | Debate, adversarial, vote, and Elite orchestration; Elite gates initial/claim-audit/revision at three successes. |
| Verdict types | [`src/conclave/verdict.py`](src/conclave/verdict.py) | Public verdict/member Pydantic types (`CouncilVerdict`, `CouncilPosition`, `CouncilConflict`, `ProviderVote`, `MinorityReport`) + the LCD JSON Schemas (`verdict_json_schema`/`member_answer_json_schema`/`verdict_extraction_json_schema`) usable across all three native structured-output surfaces; `VERDICT_SCHEMA_VERSION`/`VERDICT_EXTRACTION_PROMPT_VERSION`. |
| Agreement | [`src/conclave/agreement.py`](src/conclave/agreement.py) | Deterministic consensus: `consensus_score` (`position_cluster_ratio_v1` — largest cluster / positioned members; `None` for N<2) + `consensus_label` buckets. Pure arithmetic, no `difflib`, never LLM-emitted. |
| Verdict synthesis | [`src/conclave/verdict_synthesis.py`](src/conclave/verdict_synthesis.py) | `extract_verdict` engine: one initial extraction call and at most one repair, native `output_contract` enforcement + prompt-level fallback, validate → repair-once → graceful `verdict=None`; the three verdict-absent reasons; provenance on every return path. |
Expand All @@ -65,7 +65,7 @@ Package root: `src/conclave/` (installed as the `conclave` package; console scri
| Registry | [`src/conclave/registry.py`](src/conclave/registry.py) | Friendly-name → model-id defaults; provider → env-var mapping; key **presence** logic (never values). |
| Config | [`src/conclave/config.py`](src/conclave/config.py) | Loads/merges `~/.conclave/config.yml` over defaults; resolves model ids and named/CSV councils; parses the `endpoints:` section (custom OpenAI-compatible providers). |
| Models | [`src/conclave/models.py`](src/conclave/models.py) | Stable Pydantic contract, including `EliteResult` phase artifacts and backward-compatible `CouncilResult.elite`. |
| CLI | [`src/conclave/cli.py`](src/conclave/cli.py) | Five released `conclave ask` modes plus source-only Elite and `providers`; Elite exits 1 unless decision readiness is `ready`, emits full JSON before failure, and rejects streaming. |
| CLI | [`src/conclave/cli.py`](src/conclave/cli.py) | Six released `conclave ask` modes plus `providers`; Elite exits 1 unless decision readiness is `ready`, emits full JSON before failure, and rejects streaming. |
| Experimental evals | [`src/conclave/evals/`](src/conclave/evals/) | Versioned DSE-708 planning, strict replay, failure-inclusive running, blinding, scoring, and exploratory reports; no product-quality claim. |
| Eval CLI | [`src/conclave/eval_cli.py`](src/conclave/eval_cli.py) | Offline `plan/run/blind/report` plus `eval live`; paid execution requires `--execute`, exact USD 10.00 approval, and an owner-only `--checkpoint-seal-key-file`. |
| Logging | [`src/conclave/logging.py`](src/conclave/logging.py) | Logger factory; stderr; verbosity via `CONCLAVE_LOG_LEVEL` (default `WARNING`). |
Expand Down Expand Up @@ -97,7 +97,7 @@ Run: `pytest` (config in `pyproject.toml`, `asyncio_mode = "auto"`).
|------|------|---------|
| Packaging | [`pyproject.toml`](pyproject.toml) | hatchling build, deps (httpx, pydantic, rich, typer, pyyaml — no LLM SDK), dev extras, console script, pytest config. License: MIT. **PyPI distribution name `conclave-cli`** (the name `conclave` is an unrelated project); command + import stay `conclave`. |
| Release runbook | [`RELEASING.md`](RELEASING.md) | Operator runbook: one-time PyPI OIDC Trusted-Publisher setup for `conclave-cli`, cut-a-release checklist (bump→tag→publish Release), post-release verification (Sigstore bundle, PEP 740 attestations), rollback/yank. |
| Changelog | [`CHANGELOG.md`](CHANGELOG.md) | Keep-a-Changelog history. `[Unreleased]` includes Elite and does not claim a version or publication. |
| Changelog | [`CHANGELOG.md`](CHANGELOG.md) | Keep-a-Changelog history through v1.2.0. |
| Dev lockfile | [`requirements-dev.lock`](requirements-dev.lock) | Hash-pinned dev + runtime tree for reproducible installs/CI. Regenerate via `uv pip compile --universal --generate-hashes --python-version 3.11 --extra dev pyproject.toml -o requirements-dev.lock`. |
| License | [`LICENSE`](LICENSE) | MIT License. Copyright (c) 2026 Ernest Provo. Matches the `pyproject.toml` license field. |
| Security policy | [`SECURITY.md`](SECURITY.md) | BYO-keys vulnerability reporting policy: how to report, scope, and the key-handling guarantees consumers can rely on. |
Expand All @@ -116,6 +116,7 @@ Run: `pytest` (config in `pyproject.toml`, `asyncio_mode = "auto"`).

| Date | Change |
|------|--------|
| 2026-07-18 | Released v1.2.0: Elite, constrained-choice vote, every-mode manifest hardening, the capped paid-exploratory evaluation runner, strict OpenAI-compatible verdict extraction, and org-repository release identities. The live lane remains correctness-only; no quality or efficiency claim. |
| 2026-07-18 | Implemented the sequential paid exploratory runner: default dry-run, exact USD 10.00 execute approval, reservation-before-call, one in-flight call, and no-repeat resume. The 24-task fixture remains offline/open-book and is not the paid smoke corpus; the smoke is correctness-only and not decision eligible. |
| 2026-07-17 | Added the experimental DSE-708 offline evaluation substrate and `conclave eval` artifact workflow; no live study or quality claim. |
| 2026-07-17 | Reframed the product roadmap around empirically proven decision quality: narrowed current claims to execution traceability, identified answer IDs as internal provenance rather than external evidence, gated Elite merge on H0 correctness, and added H1-H4 evidence, buyer, and outcome gates. |
Expand Down
Loading