Skip to content

docs: close out v1.2 release and pilot status #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ernestprovo23 merged 1 commit into from
Jul 19, 2026
Merged

docs: close out v1.2 release and pilot status #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions DOCUMENTATION_INDEX.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ the canonical authority spec on top of those.

- **Repo:** `/Users/ernestprovo/dev/conclave/`
- **Version:** 1.2.0 · **License:** MIT
- **Last updated:** 2026-07-18
- **Last updated:** 2026-07-19

---

Expand All @@ -29,7 +29,7 @@ the canonical authority spec on top of those.

| Doc | Path | Purpose |
|-----|------|---------|
| **Decision Quality Roadmap** | [`docs/plans/2026-07-17-decision-quality-roadmap.md`](docs/plans/2026-07-17-decision-quality-roadmap.md) | H0-H4 prove-it roadmap: Elite correctness, randomized ablations, source-grounded Decision Briefs, buyer pull, outcome learning, old-backlog disposition, demand gates, and kill criteria. |
| **Decision Quality Roadmap** | [`docs/plans/2026-07-17-decision-quality-roadmap.md`](docs/plans/2026-07-17-decision-quality-roadmap.md) | H0-H4 prove-it roadmap: released Elite baseline, active H1 exploratory pilot, randomized ablations, source-grounded Decision Briefs, buyer pull, outcome learning, demand gates, and kill criteria. |
| **H1 Evaluation Design** | [`docs/plans/2026-07-17-h1-budget-matched-evaluation-design.md`](docs/plans/2026-07-17-h1-budget-matched-evaluation-design.md) | DSE-708 experimental boundary, six frozen conditions, replay, blinding, and failure-inclusive analysis. |
| **H1 Evaluation Plan** | [`docs/plans/2026-07-17-h1-budget-matched-evaluation.md`](docs/plans/2026-07-17-h1-budget-matched-evaluation.md) | TDD delivery plan for the offline, budget-matched evaluation substrate. |
| **H1 Method Hardening** | [`docs/plans/2026-07-17-h1-method-hardening-design.md`](docs/plans/2026-07-17-h1-method-hardening-design.md) | Paid-study provenance, task-clustered inference, grader controls, confirmatory refusal boundaries, and QA design. |
Expand Down Expand Up @@ -116,6 +116,7 @@ Run: `pytest` (config in `pyproject.toml`, `asyncio_mode = "auto"`).

| Date | Change |
|------|--------|
| 2026-07-19 | Reconciled the decision-quality roadmap to released v1.2.0 and active H1 pilot preparation. Recorded the 12-cell paid correctness smoke, isolated three-provider connectivity check, synthetic-pack evidence boundary, and two-human-grader calibration plan. Decision quality and efficiency remain unvalidated. |
| 2026-07-18 | Released v1.2.0: Elite, constrained-choice vote, every-mode manifest hardening, the capped paid-exploratory evaluation runner, strict OpenAI-compatible verdict extraction, and org-repository release identities. The live lane remains correctness-only; no quality or efficiency claim. |
| 2026-07-18 | Implemented the sequential paid exploratory runner: default dry-run, exact USD 10.00 execute approval, reservation-before-call, one in-flight call, and no-repeat resume. The 24-task fixture remains offline/open-book and is not the paid smoke corpus; the smoke is correctness-only and not decision eligible. |
| 2026-07-17 | Added the experimental DSE-708 offline evaluation substrate and `conclave eval` artifact workflow; no live study or quality claim. |
Expand Down
27 changes: 19 additions & 8 deletions docs/plans/2026-07-17-decision-quality-roadmap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Conclave Decision Quality Roadmap

**Status:** Approved direction; Horizon 0 implemented on open PR #51, pending final gate review
**Status:** Active; Horizon 0 released in v1.2.0, Horizon 1 exploratory pilot preparation underway
**Date:** 2026-07-17
**Current product:** v1.1.0 stable; Elite implemented but unreleased on open PR #51
**Current product:** v1.2.0 stable on PyPI and GitHub; Elite is released, while
decision-quality and efficiency claims remain unvalidated

## Thesis

Expand All @@ -28,10 +29,9 @@ grounding and deterministic citation validation ship, the honest claim is
- Build the narrowest workflow that improves consequential decisions; integrate commodity
infrastructure instead of recreating it.

## Horizon 0 — verify Elite correctness before merge
## Horizon 0 — Elite correctness and release gate (complete)

Elite remains implemented but unreleased. Open PR #51 implements these contracts and must
remain unmerged until the complete merge gate verifies them:
Elite shipped in v1.2.0 after the release gate verified these baseline contracts:

1. **Persistent identities.** Each initial answer has a stable identifier that survives claim
audit, revision, synthesis, serialization, and cache replay. Do not present those IDs as
Expand All @@ -51,9 +51,11 @@ remain unmerged until the complete merge gate verifies them:
must say execution-traceable. Answer IDs prove
provenance within a run; they do not prove truth.

**Merge gate:** all six items have tests, the complete suite and static checks pass, a
secret scan passes, documentation matches behavior, and PR review finds no false quality or
auditability claim. Otherwise keep Elite unreleased.
**Gate outcome:** all six items have regression coverage; the complete suite, static checks,
secret scan, package build/install, release review, and documentation checks passed. A capped
12-cell paid smoke completed every planned cell, and a separate three-provider connectivity
smoke passed from an isolated runner using the published package. These results validate
execution correctness only, not comparative quality or efficiency.

## Horizon 1 — prove that the protocol improves decisions

Expand All @@ -62,6 +64,15 @@ single-frontier-model, ordinary synthesis, adversarial, and Elite conditions usi
task set and matched token or dollar budgets. Randomize condition order and blind human
graders to mode and provider identity.

### Current H1 position (2026-07-19)

The offline harness, blinding/scoring workflow, USD 10 execution ceiling, authenticated
checkpoints, and paid correctness smoke are complete. The 24-task open-book QA pack remains
synthetic and offline-only evidence. The next valid step is a separately frozen 20-30 task
private exploratory pilot with protected grader keys, six matched conditions, two independent
human graders, and adjudication. Grader calibration is staged before the full pilot. DSE-690
and DSE-708 remain open until pilot evidence supports a go, redesign, or kill decision.

### Program stages

1. **Pilot:** 20-30 diverse, answerable decision tasks to debug rubrics, establish grader
Expand Down