From 7ca5ef0e72c4cffdc9160e73d5379de616b14560 Mon Sep 17 00:00:00 2001 From: ernestprovo23 Date: Sun, 19 Jul 2026 11:58:25 -0400 Subject: [PATCH] docs: close out v1.2 release and pilot status --- DOCUMENTATION_INDEX.md | 5 ++-- .../2026-07-17-decision-quality-roadmap.md | 27 +++++++++++++------ 2 files changed, 22 insertions(+), 10 deletions(-) diff --git a/DOCUMENTATION_INDEX.md b/DOCUMENTATION_INDEX.md index 86a5e54..0f1dd80 100644 --- a/DOCUMENTATION_INDEX.md +++ b/DOCUMENTATION_INDEX.md @@ -7,7 +7,7 @@ the canonical authority spec on top of those. - **Repo:** `/Users/ernestprovo/dev/conclave/` - **Version:** 1.2.0 ยท **License:** MIT -- **Last updated:** 2026-07-18 +- **Last updated:** 2026-07-19 --- @@ -29,7 +29,7 @@ the canonical authority spec on top of those. | Doc | Path | Purpose | |-----|------|---------| -| **Decision Quality Roadmap** | [`docs/plans/2026-07-17-decision-quality-roadmap.md`](docs/plans/2026-07-17-decision-quality-roadmap.md) | H0-H4 prove-it roadmap: Elite correctness, randomized ablations, source-grounded Decision Briefs, buyer pull, outcome learning, old-backlog disposition, demand gates, and kill criteria. | +| **Decision Quality Roadmap** | [`docs/plans/2026-07-17-decision-quality-roadmap.md`](docs/plans/2026-07-17-decision-quality-roadmap.md) | H0-H4 prove-it roadmap: released Elite baseline, active H1 exploratory pilot, randomized ablations, source-grounded Decision Briefs, buyer pull, outcome learning, demand gates, and kill criteria. | | **H1 Evaluation Design** | [`docs/plans/2026-07-17-h1-budget-matched-evaluation-design.md`](docs/plans/2026-07-17-h1-budget-matched-evaluation-design.md) | DSE-708 experimental boundary, six frozen conditions, replay, blinding, and failure-inclusive analysis. | | **H1 Evaluation Plan** | [`docs/plans/2026-07-17-h1-budget-matched-evaluation.md`](docs/plans/2026-07-17-h1-budget-matched-evaluation.md) | TDD delivery plan for the offline, budget-matched evaluation substrate. | | **H1 Method Hardening** | [`docs/plans/2026-07-17-h1-method-hardening-design.md`](docs/plans/2026-07-17-h1-method-hardening-design.md) | Paid-study provenance, task-clustered inference, grader controls, confirmatory refusal boundaries, and QA design. | @@ -116,6 +116,7 @@ Run: `pytest` (config in `pyproject.toml`, `asyncio_mode = "auto"`). | Date | Change | |------|--------| +| 2026-07-19 | Reconciled the decision-quality roadmap to released v1.2.0 and active H1 pilot preparation. Recorded the 12-cell paid correctness smoke, isolated three-provider connectivity check, synthetic-pack evidence boundary, and two-human-grader calibration plan. Decision quality and efficiency remain unvalidated. | | 2026-07-18 | Released v1.2.0: Elite, constrained-choice vote, every-mode manifest hardening, the capped paid-exploratory evaluation runner, strict OpenAI-compatible verdict extraction, and org-repository release identities. The live lane remains correctness-only; no quality or efficiency claim. | | 2026-07-18 | Implemented the sequential paid exploratory runner: default dry-run, exact USD 10.00 execute approval, reservation-before-call, one in-flight call, and no-repeat resume. The 24-task fixture remains offline/open-book and is not the paid smoke corpus; the smoke is correctness-only and not decision eligible. | | 2026-07-17 | Added the experimental DSE-708 offline evaluation substrate and `conclave eval` artifact workflow; no live study or quality claim. | diff --git a/docs/plans/2026-07-17-decision-quality-roadmap.md b/docs/plans/2026-07-17-decision-quality-roadmap.md index fd4e95b..ff9daf7 100644 --- a/docs/plans/2026-07-17-decision-quality-roadmap.md +++ b/docs/plans/2026-07-17-decision-quality-roadmap.md @@ -1,8 +1,9 @@ # Conclave Decision Quality Roadmap -**Status:** Approved direction; Horizon 0 implemented on open PR #51, pending final gate review +**Status:** Active; Horizon 0 released in v1.2.0, Horizon 1 exploratory pilot preparation underway **Date:** 2026-07-17 -**Current product:** v1.1.0 stable; Elite implemented but unreleased on open PR #51 +**Current product:** v1.2.0 stable on PyPI and GitHub; Elite is released, while +decision-quality and efficiency claims remain unvalidated ## Thesis @@ -28,10 +29,9 @@ grounding and deterministic citation validation ship, the honest claim is - Build the narrowest workflow that improves consequential decisions; integrate commodity infrastructure instead of recreating it. -## Horizon 0 โ€” verify Elite correctness before merge +## Horizon 0 โ€” Elite correctness and release gate (complete) -Elite remains implemented but unreleased. Open PR #51 implements these contracts and must -remain unmerged until the complete merge gate verifies them: +Elite shipped in v1.2.0 after the release gate verified these baseline contracts: 1. **Persistent identities.** Each initial answer has a stable identifier that survives claim audit, revision, synthesis, serialization, and cache replay. Do not present those IDs as @@ -51,9 +51,11 @@ remain unmerged until the complete merge gate verifies them: must say execution-traceable. Answer IDs prove provenance within a run; they do not prove truth. -**Merge gate:** all six items have tests, the complete suite and static checks pass, a -secret scan passes, documentation matches behavior, and PR review finds no false quality or -auditability claim. Otherwise keep Elite unreleased. +**Gate outcome:** all six items have regression coverage; the complete suite, static checks, +secret scan, package build/install, release review, and documentation checks passed. A capped +12-cell paid smoke completed every planned cell, and a separate three-provider connectivity +smoke passed from an isolated runner using the published package. These results validate +execution correctness only, not comparative quality or efficiency. ## Horizon 1 โ€” prove that the protocol improves decisions @@ -62,6 +64,15 @@ single-frontier-model, ordinary synthesis, adversarial, and Elite conditions usi task set and matched token or dollar budgets. Randomize condition order and blind human graders to mode and provider identity. +### Current H1 position (2026-07-19) + +The offline harness, blinding/scoring workflow, USD 10 execution ceiling, authenticated +checkpoints, and paid correctness smoke are complete. The 24-task open-book QA pack remains +synthetic and offline-only evidence. The next valid step is a separately frozen 20-30 task +private exploratory pilot with protected grader keys, six matched conditions, two independent +human graders, and adjudication. Grader calibration is staged before the full pilot. DSE-690 +and DSE-708 remain open until pilot evidence supports a go, redesign, or kill decision. + ### Program stages 1. **Pilot:** 20-30 diverse, answerable decision tasks to debug rubrics, establish grader