From 17d28a4a02e5c21b40555903b8cf1b66ba2278dc Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:38:44 +0930 Subject: [PATCH 01/66] docs: add Codex agent guidance --- AGENTS.md | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..2fd6cd1 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,65 @@ +# Agent instructions for coco + +This repository builds `coco`, a small loop-engineering referee for AI coding agents. The core product boundary is intentionally strict: the agent may plan, implement, review, and verify through the referee, but merge authority stays explicit and goal-scoped. + +## Project shape + +- `src/commands/**` contains CLI/domain command entry points. +- `src/gate.ts`, `src/epoch.ts`, `src/state.ts`, `src/git.ts`, `src/lock.ts`, `src/oracleVerdict.ts`, `src/commands/goal*.ts`, `src/commands/verify.ts`, and `src/commands/merge.ts` are referee-critical. +- `src/mcp/**` exposes the same referee through MCP tools. +- `src/store/**` is the PM/knowledge layer. It must not import or mutate goal/referee state. +- `src/improve/**` implements propose-only self-improvement guards and audit-derived signals. +- `skills/**` contains the user-facing Codex/Claude skills. +- `.coco/**` is local runtime state and must not be committed. +- `.coco-store/**` is the local PM store. Treat audit-derived/self-improvement content as local unless code explicitly marks it shared. + +## Commands + +Use pnpm. + +```sh +pnpm install +pnpm typecheck +pnpm test +pnpm build +pnpm ci +``` + +`pnpm ci` is the expected pre-PR verification command. If a change touches a safety-critical gate, run the most specific affected tests as well as `pnpm ci`. + +## Safety invariants + +Do not weaken these without an explicit, human-authored referee-change goal and tests that prove the new behavior: + +1. A review verdict must come from strict Oracle output parsing; never add a caller-asserted clean verdict path. +2. Verify is coco-owned. The agent must not self-report `pass`. +3. Review and verify are bound to the current HEAD/tree and require a clean working tree. +4. A tree that was blocking/failing must not be silently re-blessed without new code and fresh review/verify. +5. Merge requires active goal, correct branch, clean tree, implement in the current epoch, clean review, passing verify, and base ancestry. +6. Auto-merge is opt-in per goal and must never loosen the human merge path. +7. Improve-origin changes must not touch protected referee, metrics, store, or improve-self paths. +8. Store/pack must preserve the privacy boundary: local cards and secret-looking files must not be sent to Oracle. + +## Coding style + +- Keep domain rules deterministic and testable. Prefer pure derivation functions for state-machine facts. +- Keep LLM/Oracle assumptions in skills or explicit integration seams, not in the referee core. +- Fail closed at trust boundaries: malformed state, ambiguous verdicts, missing verify config, path traversal, stale/forged verify runs, and protected-path changes should stop the loop rather than infer success. +- Avoid broad rewrites of `skills/**`; make small, reviewable deltas because skill text is part of the safety surface. +- Add regression tests for every bug fix. For referee changes, include the concrete false-green or unsafe-transition scenario. + +## Branch and merge expectations + +- Work on a branch; do not commit directly to `main`. +- Do not merge from an agent session unless the user explicitly approves the exact merge action for that goal/branch. +- A ready PR should explain the safety impact, affected commands/tools, and verification run. + +## Daily Codex workflow + +For normal development in Codex.app: + +1. Run `coco doctor` before a long session. +2. Use `$coco-goal` for vague or multi-step work. +3. Use `$coco-loop` for one loop-sized implementation task. +4. Use `$coco-store status` or `coco-store progress` for project state. +5. Use `$coco-improve` only to propose one evidence-backed improvement; it must not build, edit, or merge the proposal itself. From 78c6ffe3fed4cc0b9a6b9bea25250cd7788dd91d Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:39:08 +0930 Subject: [PATCH 02/66] docs: add daily Codex workflow --- docs/codex-daily-workflow.md | 105 +++++++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 docs/codex-daily-workflow.md diff --git a/docs/codex-daily-workflow.md b/docs/codex-daily-workflow.md new file mode 100644 index 0000000..29a303f --- /dev/null +++ b/docs/codex-daily-workflow.md @@ -0,0 +1,105 @@ +# Daily Codex.app workflow + +This guide is the practical path for using coco every day from Codex.app. + +## One-time setup + +1. Install dependencies and build locally: + + ```sh + pnpm install + pnpm build + ``` + +2. Configure the coco MCP server in Codex. Start from `.codex/config.toml.example` and copy the relevant blocks into `~/.codex/config.toml`. + +3. Install the skills in your agent skills directory. Keep the installed skills in sync with this repository when developing coco itself. + +4. Run: + + ```sh + coco doctor + ``` + + Fix warnings that would block the full loop, especially missing `verify.testCommand`, missing MCP registration, or missing Oracle wiring. + +## Start of session + +Run: + +```sh +coco doctor +coco-store status +``` + +Use the doctor output to catch broken local wiring before starting a long loop. Use the store status card to see backlog/spec state without reading the whole repository. + +## Turning intent into work + +For vague or multi-step work, start with: + +```text +$coco-goal +``` + +The goal skill should read the repo, research current external constraints where relevant, produce a strong GoalSpec, archive it as a `coco-store` spec, promote loop-sized tasks to `BACKLOG.md`, and stop. + +## Building one task + +For one implementation task, use: + +```text +$coco-loop +``` + +or, when the backlog has ready tasks: + +```text +$coco-loop +``` + +Use `--auto` only when you intentionally grant forward consent for this one goal: + +```text +$coco-loop --auto +``` + +Auto-merge is still gated by clean review, coco-owned verify, base ancestry, risk policy, and per-goal consent. Risk fallback returns to the human merge path. + +## Human merge checkpoint + +The normal loop ends at `merge-gate`. The agent should present the exact command: + +```sh +coco merge --goal +``` + +Approving that exact command is the merge consent. Do not approve blanket or always-allow merge behavior. + +## Self-improvement + +Run: + +```text +$coco-improve +``` + +only when there is enough audit history. The improve skill should propose exactly one local improve-spec and one non-protected backlog task, then stop. It must not edit live code, start a loop, or merge. + +## Pre-PR verification + +Before a branch is ready for review, run: + +```sh +pnpm ci +``` + +For referee-critical changes, also run targeted tests for the exact invariant being changed or protected. + +## Troubleshooting checklist + +- `coco doctor` reports `oracle MCP` missing: configure Oracle before plan/review gates. +- `goal status` warns about `verify.testCommand`: set `verify.testCommand` in committed `coco.config.json`. +- `health` reports `review-unavailable`: fix Oracle login/wiring, then resume with `coco_goal_op_clear`. +- `health` reports `stalled`: inspect the current `nextAction`; do not guess the phase from chat memory. +- Verify is stuck running: use `coco doctor clean` for old terminal/orphaned verify runs; do not delete live goal state. From e541e2b0c84d7d0dba3f4c7a6d3982fc69784ea6 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:39:17 +0930 Subject: [PATCH 03/66] docs: add Codex MCP config example --- .codex/config.toml.example | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 .codex/config.toml.example diff --git a/.codex/config.toml.example b/.codex/config.toml.example new file mode 100644 index 0000000..9b04a75 --- /dev/null +++ b/.codex/config.toml.example @@ -0,0 +1,14 @@ +# Copy the relevant blocks into ~/.codex/config.toml. +# Keep this file secret-free: never put tokens, cookies, or browser profile data here. + +[mcp_servers.coco] +command = "coco-mcp" +args = [] + +# Oracle is required for the full coco-loop plan/review gate. +# Replace command/args with the Oracle MCP entrypoint you installed locally. +# Leave this block out until your Oracle setup is verified. +# +# [mcp_servers.oracle] +# command = "oracle" +# args = ["mcp"] From 951c366d0e3f758ef0b4b0cd0322963c300650d8 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:39:32 +0930 Subject: [PATCH 04/66] ci: add typecheck test build workflow --- .github/workflows/ci.yml | 44 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 .github/workflows/ci.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..84210c2 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,44 @@ +name: CI + +on: + pull_request: + push: + branches: [main] + +permissions: + contents: read + +jobs: + verify: + name: Node ${{ matrix.node }} on ${{ matrix.os }} + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest] + node: [20, 22] + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Enable Corepack + run: corepack enable + + - name: Setup Node + uses: actions/setup-node@v4 + with: + node-version: ${{ matrix.node }} + cache: pnpm + + - name: Install dependencies + run: pnpm install --frozen-lockfile + + - name: Typecheck + run: pnpm typecheck + + - name: Test + run: pnpm test + + - name: Build + run: pnpm build From 6e6fd273eceb2e646d53a3aa42f8edfbaa14c9a1 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:39:57 +0930 Subject: [PATCH 05/66] chore: expose mcp bin and add ci scripts --- package.json | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/package.json b/package.json index 8ed9de2..fa70c59 100644 --- a/package.json +++ b/package.json @@ -22,6 +22,7 @@ ], "bin": { "coco": "dist/coco.js", + "coco-mcp": "dist/coco-mcp.js", "coco-store": "dist/coco-store.js" }, "files": [ @@ -29,16 +30,18 @@ "skills" ], "engines": { - "node": ">=18" + "node": ">=20" }, "publishConfig": { "access": "public" }, "scripts": { "build": "tsdown", + "typecheck": "tsc --noEmit", "test": "vitest run", + "ci": "pnpm typecheck && pnpm test && pnpm build", "dev": "tsx src/bin/coco.ts", - "prepublishOnly": "pnpm build" + "prepublishOnly": "pnpm ci" }, "devDependencies": { "@types/node": "^22.0.0", From 6264e9101dfc6b18bc6fc08fad0bbc0ba9ec3404 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:41:13 +0930 Subject: [PATCH 06/66] docs: document Codex daily setup and verification --- README.md | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c319c73..d329e85 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,9 @@ As of **v0.4**, coco also **improves itself**: `coco-audit` records every meanin ## What's new +- **Daily Codex hardening** — root `AGENTS.md` now captures durable repo/agent guidance, `.codex/config.toml.example` shows the local MCP wiring, and `docs/codex-daily-workflow.md` gives the Mac Codex.app daily path. +- **`coco-mcp` package bin** — the npm package now exposes `coco-mcp` alongside `coco` and `coco-store`, matching the documented MCP setup path. +- **CI-ready verification** — `pnpm typecheck`, `pnpm ci`, and a GitHub Actions matrix run typecheck + tests + build on Linux and macOS. - **`coco-improve` — self-improving loop (propose-only)** — `$coco-improve` reflects over the `coco audit` corpus (with an **insufficient-data gate** so a thin history can't manufacture findings), forms ONE incremental, evidence-backed hypothesis, and archives it as a **local** improve-spec + one loop-sized backlog task for a human to run through `$coco-loop`. It never builds, merges, or edits live skills. It can ground a proposal in **bounded, cited web research** — a *code-controlled* query, so nothing about coco leaks into a search — treating any external best-practice as *evidence to test, never proof*. A **code-owned guard** (`coco improve check`) refuses any change targeting the referee / metrics / store — enforced at BOTH propose time and, via an `improveOrigin` flag **frozen at goal start**, at **merge** time (even if a task under-declares what it touches). A self-improvement can never quietly weaken coco's own gate. - **`coco-audit` — automatic trajectory capture** — every meaningful loop/goal action (reviews, fixes, verify results, Oracle outages, merges) is recorded to a local, gitignored `.coco/audit.ndjson` at the domain chokepoint. Deterministic, **best-effort** (a logging failure never breaks the referee), and **redacted** (structural facts only — no evidence text). `coco audit report` aggregates it: fix-rounds (by distinct blocking tree, matching the epoch model), verify failures, Oracle outages, and verify→merge latency — the signal for evolving the loop. - **`coco doctor` — one-shot health check** — a read-only, **no-LLM** diagnostic that aggregates environment (node/git/version), repo setup (init, `verify.testCommand`), wiring (merge-guard hooks, coco + Oracle MCP, watchdog), active-goal health, and data hygiene. `coco doctor clean` reclaims stale verify-run cache — **dry-run by default**, `--apply` to delete, and only terminal/orphaned runs (never a live goal's runs, the goal ledger, or the audit log). @@ -35,7 +38,7 @@ As of **v0.4**, coco also **improves itself**: `coco-audit` records every meanin ## Prerequisites -- **Node.js ≥ 18.** +- **Node.js ≥ 20.** - **An MCP-aware coding agent** — [OpenAI Codex](https://developers.openai.com/codex) or [Claude Code](https://claude.ai/code) — with the `coco` (and `oracle`) MCP servers registered and the `coco-goal` / `coco-store` / `coco-loop` skills installed under the agent's skills dir. - **A ChatGPT Pro subscription + Oracle — required for the review brain.** coco's plan/review gate runs **GPT‑5.x‑Pro** through the [`@steipete/oracle`](https://github.com/steipete/oracle) lane, which drives a logged-in **ChatGPT Pro** browser session (the `consult` MCP tool). Without an active **ChatGPT Pro** subscription and Oracle configured, the CLIs still run, but the Oracle-gated plan/review steps can't — by design coco then **fails to the human, never a false green**. @@ -47,7 +50,30 @@ As of **v0.4**, coco also **improves itself**: `coco-audit` records every meanin npm install -g @nickcao/coco ``` -Provides `coco` (the referee), `coco-store` (the PM layer), and the `coco-mcp` stdio server for MCP-aware agents. +Provides `coco` (the referee), `coco-store` (the PM layer), and `coco-mcp` (the MCP stdio server for MCP-aware agents). + +## Daily Codex.app setup + +For Codex.app, start from the durable guidance and examples in this repo: + +```sh +cat AGENTS.md +cat .codex/config.toml.example +cat docs/codex-daily-workflow.md +``` + +Then copy the relevant MCP blocks into `~/.codex/config.toml`, install/sync the skills into your agent skills directory, and run: + +```sh +coco doctor +``` + +The daily path is: + +1. `$coco-goal ` for vague or multi-step work. +2. `$coco-loop` for one ready backlog task, or `$coco-loop ` for a specific task. +3. `coco-store status` / `coco-store progress` for PM state. +4. `$coco-improve` only to propose one evidence-backed improvement; it never edits or merges the proposal itself. ## Quick taste (CLI, no agent) @@ -64,6 +90,18 @@ coco improve digest # audit-derived pain signals (insufficien coco improve check # refuse edits to the referee/metrics/store (exit 3) ``` +## Development verification + +```sh +pnpm install +pnpm typecheck +pnpm test +pnpm build +pnpm ci +``` + +`pnpm ci` is the pre-PR gate: typecheck, test, then build. GitHub Actions runs the same gate on Linux and macOS. + ## Credits - **Review + plan brain — [Oracle (`@steipete/oracle`)](https://github.com/steipete/oracle)** by [Peter Steinberger](https://github.com/steipete): the lane that drives **ChatGPT Pro (GPT‑5.x‑Pro)** for deep review, planning, and research. coco's "never false-green" gate is only as strong as this brain — coco leans on it at every plan and review step, and wouldn't exist without it. (Requires your own ChatGPT Pro subscription.) From 5ae9dd349aae85c5754050b7210a39f49f107d57 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:51:13 +0930 Subject: [PATCH 07/66] feat: version and validate goal state shape --- src/types.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/types.ts b/src/types.ts index 15a69ae..b24ca81 100644 --- a/src/types.ts +++ b/src/types.ts @@ -23,10 +23,11 @@ export interface InFlight { } export interface GoalState { + schemaVersion?: number; // persisted goal-ledger schema. Missing on old goals; readGoal migrates to the current version. id: string; objective: string; branch: string; - base: string; // e.g. "main" + base: string; // e.g. "main" or a configured base branch baseTree?: string; // git tree hash of the base at goal start — an implement must differ from it (no no-op work) state: GoalLifecycle; maxFixRounds: number; From 8c9d124b2d8aad90924036ef190f35a3a4d48680 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:51:48 +0930 Subject: [PATCH 08/66] feat: add persisted goal schema parser --- src/goalSchema.ts | 113 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 src/goalSchema.ts diff --git a/src/goalSchema.ts b/src/goalSchema.ts new file mode 100644 index 0000000..f5e330a --- /dev/null +++ b/src/goalSchema.ts @@ -0,0 +1,113 @@ +import { ZodError, z } from 'zod'; +import type { GoalState } from './types.js'; + +export const GOAL_SCHEMA_VERSION = 1; + +const phaseSchema = z.enum(['plan', 'implement', 'review', 'verify']); +const verdictSchema = z.enum(['clean', 'blocking', 'pass', 'fail']); +const lifecycleSchema = z.enum(['active', 'achieved', 'blocked', 'failed', 'cancelled']); + +const goalEventSchema = z + .object({ + phase: phaseSchema, + at: z.string().min(1), + commit: z.string().min(1), + tree: z.string().min(1), + verdict: verdictSchema.optional(), + evidence: z.string().optional(), + runId: z.string().optional(), + }) + .passthrough(); + +const inFlightSchema = z + .object({ + phase: phaseSchema, + kind: z.enum(['oracle', 'test']), + startedAt: z.string().min(1), + runId: z.string().optional(), + }) + .passthrough(); + +const reviewUnavailableSchema = z + .object({ + at: z.string().min(1), + phase: z.enum(['plan', 'review']), + commit: z.string().min(1), + tree: z.string().min(1), + reason: z.enum(['preflight-failed', 'oracle-timeout', 'oracle-error', 'ambiguous-verdict']), + attempts: z.number().int().positive(), + evidence: z.string().optional(), + }) + .passthrough(); + +const failureLoopSchema = z + .object({ + key: z.string().min(1), + count: z.number().int().nonnegative(), + history: z.array( + z + .object({ + key: z.string().min(1), + at: z.string().min(1), + commit: z.string().min(1), + tree: z.string().min(1), + }) + .passthrough(), + ), + }) + .passthrough(); + +const goalStateSchema = z + .object({ + schemaVersion: z.number().int().positive().optional(), + id: z.string().min(1), + objective: z.string().min(1), + branch: z.string().min(1), + base: z.string().min(1), + baseTree: z.string().optional(), + state: lifecycleSchema, + maxFixRounds: z.number().int().positive(), + acceptanceChecks: z.array(z.string()), + events: z.array(goalEventSchema), + createdAt: z.string().optional(), + updatedAt: z.string().optional(), + lastActivityAt: z.string().optional(), + lastOperation: z.string().optional(), + backlogTaskId: z.string().optional(), + autoMergeAllowed: z.boolean().optional(), + improveOrigin: z.boolean().optional(), + budget: z.object({ maxWallClockMin: z.number().positive().optional() }).passthrough().optional(), + inFlight: inFlightSchema.optional(), + failureLoop: failureLoopSchema.optional(), + reviewUnavailable: reviewUnavailableSchema.optional(), + }) + .passthrough(); + +function explainZod(e: ZodError): string { + return e.issues.map((i) => `${i.path.join('.') || ''}: ${i.message}`).join('; '); +} + +/** Parse and migrate persisted `.coco/goals/*.json`. This is a trust boundary: malformed state must + * become `invalid-state` / a clear command error, never a partially-trusted GoalState cast. */ +export function parseGoalState(raw: unknown): GoalState { + try { + const parsed = goalStateSchema.parse(raw) as GoalState; + return { ...parsed, schemaVersion: parsed.schemaVersion ?? GOAL_SCHEMA_VERSION }; + } catch (e) { + if (e instanceof ZodError) throw new Error(`coco: goal state schema invalid (${explainZod(e)})`); + throw e; + } +} + +export function parseGoalFile(raw: string): GoalState { + try { + return parseGoalState(JSON.parse(raw) as unknown); + } catch (e) { + if (e instanceof SyntaxError) throw new Error(`coco: goal JSON parse failed (${e.message})`); + throw e; + } +} + +export function stampGoalSchema(goal: GoalState): GoalState { + return { ...goal, schemaVersion: goal.schemaVersion ?? GOAL_SCHEMA_VERSION }; +} From 11a4491ac182cee2b2835d44f12fbff6f6539a37 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:52:05 +0930 Subject: [PATCH 09/66] feat: validate and stamp goal ledgers --- src/state.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/state.ts b/src/state.ts index 1295f64..02dd79c 100644 --- a/src/state.ts +++ b/src/state.ts @@ -2,6 +2,7 @@ import { existsSync, readdirSync, readFileSync, renameSync, writeFileSync } from import { randomBytes } from 'node:crypto'; import { join } from 'node:path'; import { auditGoalWrite } from './audit.js'; +import { parseGoalFile, stampGoalSchema } from './goalSchema.js'; import { goalsDir } from './paths.js'; import type { GoalState } from './types.js'; @@ -10,13 +11,13 @@ export function goalPath(repo: string, id: string): string { } export function readGoal(path: string): GoalState { - return JSON.parse(readFileSync(path, 'utf8')) as GoalState; + return parseGoalFile(readFileSync(path, 'utf8')); } /** Atomic write: temp file + rename. */ export function writeGoal(path: string, goal: GoalState): void { const tmp = `${path}.${randomBytes(6).toString('hex')}.tmp`; - writeFileSync(tmp, `${JSON.stringify(goal, null, 2)}\n`); + writeFileSync(tmp, `${JSON.stringify(stampGoalSchema(goal), null, 2)}\n`); renameSync(tmp, path); } From 7f497804e929554a22a306a6215675634a24598c Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:53:24 +0930 Subject: [PATCH 10/66] feat: add workflow config and additive auto-merge globs --- src/cocoConfig.ts | 115 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 90 insertions(+), 25 deletions(-) diff --git a/src/cocoConfig.ts b/src/cocoConfig.ts index 3ac09d8..b886921 100644 --- a/src/cocoConfig.ts +++ b/src/cocoConfig.ts @@ -8,22 +8,34 @@ export interface VerifyConfig { outputLimitBytes?: number; } -/** Parse a `coco.config.json` body into a VerifyConfig, or null if missing/malformed. */ -function parseVerifyConfig(raw: string): VerifyConfig | null { +export interface WorkflowConfig { + baseBranch?: string; +} + +function parseJson(raw: string): unknown | null { try { - const cfg = JSON.parse(raw) as { verify?: Partial }; - const v = cfg?.verify; - if (!v || typeof v.testCommand !== 'string' || !v.testCommand.trim()) return null; - return { - testCommand: v.testCommand, - timeoutSec: typeof v.timeoutSec === 'number' && v.timeoutSec > 0 ? v.timeoutSec : undefined, - outputLimitBytes: typeof v.outputLimitBytes === 'number' && v.outputLimitBytes > 0 ? v.outputLimitBytes : undefined, - }; + return JSON.parse(raw) as unknown; } catch { return null; } } +function asObject(v: unknown): Record | null { + return v && typeof v === 'object' && !Array.isArray(v) ? (v as Record) : null; +} + +/** Parse a `coco.config.json` body into a VerifyConfig, or null if missing/malformed. */ +function parseVerifyConfig(raw: string): VerifyConfig | null { + const cfg = asObject(parseJson(raw)); + const v = asObject(cfg?.verify); + if (!v || typeof v.testCommand !== 'string' || !v.testCommand.trim()) return null; + return { + testCommand: v.testCommand, + timeoutSec: typeof v.timeoutSec === 'number' && v.timeoutSec > 0 ? v.timeoutSec : undefined, + outputLimitBytes: typeof v.outputLimitBytes === 'number' && v.outputLimitBytes > 0 ? v.outputLimitBytes : undefined, + }; +} + /** Read the TRACKED `coco.config.json` at the repo root (NOT `.coco/`, which is git-ignored runtime * state). Returns the verify config, or null if the file/field is missing or malformed. The test * command is committed repo policy — that is what makes it safe for coco to run in a shell. */ @@ -55,9 +67,51 @@ export function verifyTestCommandChange(repo: string, base: string, head = 'HEAD export const VERIFY_TEST_COMMAND_CHANGED_WARNING = 'verify.testCommand differs between base and HEAD — verify ran the branch version, so review should confirm this verification-policy change'; +export const VERIFY_TEST_COMMAND_CHANGE_ACK = + 'verify.testCommand changed in this goal. Human merge requires explicit acknowledgement: `coco merge --goal --ack-verify-policy-change`'; + export const VERIFY_NOT_CONFIGURED_WARNING = 'coco.config.json has no verify.testCommand set — coco runs this itself at the verify gate (there is no agent fallback), so configure it before you get there'; +// --- Workflow policy (coco.config.json → `workflow`) --- + +function parseWorkflowConfig(raw: string): WorkflowConfig { + const cfg = asObject(parseJson(raw)); + const w = asObject(cfg?.workflow); + const baseBranch = typeof w?.baseBranch === 'string' && w.baseBranch.trim() ? w.baseBranch.trim() : undefined; + return { ...(baseBranch ? { baseBranch } : {}) }; +} + +export function readWorkflowConfig(repo: string): WorkflowConfig { + const p = join(repo, 'coco.config.json'); + if (!existsSync(p)) return {}; + return parseWorkflowConfig(readFileSync(p, 'utf8')); +} + +function localRefExists(repo: string, ref: string): boolean { + return tryGit(repo, ['rev-parse', '--verify', '--quiet', ref]).ok; +} + +/** Resolve the branch a new goal should fork from. Config wins; otherwise prefer the repo's default + * remote branch, then common local defaults, then the current branch. */ +export function resolveBaseBranch(repo: string): string { + const configured = readWorkflowConfig(repo).baseBranch; + if (configured) return configured; + + const originHead = tryGit(repo, ['symbolic-ref', '--short', 'refs/remotes/origin/HEAD']); + if (originHead.ok) { + const b = originHead.out.trim().replace(/^origin\//, ''); + if (b) return b; + } + + for (const candidate of ['main', 'master', 'trunk', 'develop']) { + if (localRefExists(repo, candidate)) return candidate; + } + + const current = tryGit(repo, ['branch', '--show-current']); + return current.ok && current.out.trim() ? current.out.trim() : 'main'; +} + // --- Layer 2 auto-merge policy (coco.config.json → `autoMerge`) --- export interface AutoMergeConfig { @@ -94,22 +148,33 @@ function isStringArray(v: unknown): v is string[] { return Array.isArray(v) && v.every((x) => typeof x === 'string'); } -/** Parse the `autoMerge` block, merging partial user policy over defaults. Missing/malformed → defaults. */ +function uniq(xs: string[]): string[] { + return [...new Set(xs.map((x) => x.trim()).filter(Boolean))]; +} + +/** Parse the `autoMerge` block. User globs are ADDITIVE by default so a partial config cannot + * accidentally relax coco's conservative defaults; explicit `replaceDefault*Globs:true` is required + * for replacement, and `.coco/**` + `coco.config.json` still remain always-sensitive in autoMergeRisk. */ function parseAutoMergeConfig(raw: string): AutoMergeConfig { - try { - const a = (JSON.parse(raw) as { autoMerge?: Partial })?.autoMerge; - if (!a || typeof a !== 'object') return DEFAULT_AUTO_MERGE_CONFIG; - return { - maxChangedLines: - typeof a.maxChangedLines === 'number' && a.maxChangedLines > 0 - ? Math.floor(a.maxChangedLines) - : DEFAULT_AUTO_MERGE_CONFIG.maxChangedLines, - sensitiveGlobs: isStringArray(a.sensitiveGlobs) ? a.sensitiveGlobs : DEFAULT_AUTO_MERGE_CONFIG.sensitiveGlobs, - testGlobs: isStringArray(a.testGlobs) ? a.testGlobs : DEFAULT_AUTO_MERGE_CONFIG.testGlobs, - }; - } catch { - return DEFAULT_AUTO_MERGE_CONFIG; - } + const cfg = asObject(parseJson(raw)); + const a = asObject(cfg?.autoMerge); + if (!a) return DEFAULT_AUTO_MERGE_CONFIG; + + const sensitiveRaw = isStringArray(a.sensitiveGlobs) ? a.sensitiveGlobs : []; + const sensitiveAdd = isStringArray(a.additionalSensitiveGlobs) ? a.additionalSensitiveGlobs : []; + const testRaw = isStringArray(a.testGlobs) ? a.testGlobs : []; + const testAdd = isStringArray(a.additionalTestGlobs) ? a.additionalTestGlobs : []; + const replaceSensitive = a.replaceDefaultSensitiveGlobs === true; + const replaceTests = a.replaceDefaultTestGlobs === true; + + return { + maxChangedLines: + typeof a.maxChangedLines === 'number' && a.maxChangedLines > 0 + ? Math.floor(a.maxChangedLines) + : DEFAULT_AUTO_MERGE_CONFIG.maxChangedLines, + sensitiveGlobs: uniq([...(replaceSensitive ? [] : DEFAULT_AUTO_MERGE_CONFIG.sensitiveGlobs), ...sensitiveRaw, ...sensitiveAdd]), + testGlobs: uniq([...(replaceTests ? [] : DEFAULT_AUTO_MERGE_CONFIG.testGlobs), ...testRaw, ...testAdd]), + }; } /** Read the auto-merge policy as committed AT `ref` (always the goal BASE, never HEAD) so a branch From 8bae5661cf43600713251d9415981426e130be40 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:54:05 +0930 Subject: [PATCH 11/66] feat: support configurable base branch and collision-safe goal ids --- src/commands/goalStart.ts | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/src/commands/goalStart.ts b/src/commands/goalStart.ts index 9050796..3c27760 100644 --- a/src/commands/goalStart.ts +++ b/src/commands/goalStart.ts @@ -1,8 +1,10 @@ import { existsSync, readFileSync } from 'node:fs'; import { join } from 'node:path'; import { auditGoalWrite } from '../audit.js'; -import { checkout, createBranch, gatherLive, isClean, treeOfRef } from '../git.js'; +import { resolveBaseBranch } from '../cocoConfig.js'; +import { checkout, createBranch, gatherLive, isClean, treeOfRef, tryGit } from '../git.js'; import { nextAction, type NextAction } from '../gate.js'; +import { GOAL_SCHEMA_VERSION } from '../goalSchema.js'; import { isImproveOriginTask } from '../improve/originGate.js'; import { withLock } from '../lock.js'; import { findActiveGoal, goalPath, writeGoal } from '../state.js'; @@ -18,6 +20,7 @@ export interface StartOptions { maxFixRounds: number; acceptanceChecks: string[]; backlogTaskId?: string; + base?: string; // optional per-goal override; defaults to workflow.baseBranch / repo default branch autoMergeAllowed?: boolean; // forward consent for Layer 2 auto-merge (from `$coco-loop --auto` / `coco goal start --auto-merge`) budget?: GoalBudget; now?: Date; @@ -40,6 +43,21 @@ export function goalId(objective: string, now: Date): string { return `goal-${day}-${hm}-${slugify(objective)}`; } +function branchExists(repo: string, branch: string): boolean { + return tryGit(repo, ['show-ref', '--verify', '--quiet', `refs/heads/${branch}`]).ok; +} + +/** Keep the existing human-friendly minute slug for the first goal, but retry with a numeric suffix + * when a cancelled/finished goal with the same objective already left a branch or goal file behind. */ +function uniqueGoalId(repo: string, objective: string, now: Date): string { + const base = goalId(objective, now); + for (let i = 0; i < 100; i++) { + const candidate = i === 0 ? base : `${base}-${i + 1}`; + if (!existsSync(goalPath(repo, candidate)) && !branchExists(repo, `coco/${candidate}`)) return candidate; + } + throw new Error(`coco: could not allocate a unique goal id for '${base}'`); +} + export function goalStart( repo: string, opts: StartOptions, @@ -56,18 +74,20 @@ export function goalStart( if (!isClean(repo)) throw new Error('coco: working tree must be clean to start a goal'); const nowDate = opts.now ?? new Date(); - const id = goalId(opts.objective, nowDate); + const id = uniqueGoalId(repo, opts.objective, nowDate); const branch = `coco/${id}`; - const baseTree = treeOfRef(repo, 'main'); // snapshot the base tree so a no-op implement can be rejected - createBranch(repo, branch, 'main'); // branch from the main ref regardless of current HEAD + const base = opts.base ?? resolveBaseBranch(repo); + const baseTree = treeOfRef(repo, base); // snapshot the base tree so a no-op implement can be rejected + createBranch(repo, branch, base); // branch from the resolved base ref regardless of current HEAD checkout(repo, branch); const now = nowDate.toISOString(); const goal: GoalState = { + schemaVersion: GOAL_SCHEMA_VERSION, id, objective: opts.objective, branch, - base: 'main', + base, baseTree, state: 'active', maxFixRounds: opts.maxFixRounds, From 1379eed5f79e59ceb676cdfe7c46027cc95f65fb Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:55:03 +0930 Subject: [PATCH 12/66] feat: scaffold workflow base branch config --- src/commands/init.ts | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/commands/init.ts b/src/commands/init.ts index 2709358..bc83477 100644 --- a/src/commands/init.ts +++ b/src/commands/init.ts @@ -8,6 +8,7 @@ import { cocoDir, goalsDir } from '../paths.js'; * placeholder (empty → parses to no verify config → goal status surfaces a non-blocking warning * until set), so the verify gate is discoverable at init time rather than after plan/implement/review. * The `//` keys are inline doc notes (JSON has no comments); the config parser ignores unknown keys. + * `workflow.baseBranch` controls what branch new goals fork from; init-created repos use `main`. * The `autoMerge` block is the Layer 2 policy (only consulted when a goal opts in via * `coco goal start --auto-merge`); it's scaffolded at the code defaults so it's discoverable/tunable. */ const STARTER_COCO_CONFIG = { @@ -17,7 +18,11 @@ const STARTER_COCO_CONFIG = { timeoutSec: 600, outputLimitBytes: 65536, }, - '// autoMerge': 'Only used when a goal opts in (coco goal start --auto-merge). Policy is read at the goal BASE ref; auto-merge is blocked on sensitive-glob paths, diffs over maxChangedLines, or diffs with no test files. coco.config.json is ALWAYS sensitive (self-tamper guard) regardless of this list.', + '// workflow': 'New goals branch from workflow.baseBranch. If unset, coco uses the repo default branch (origin/HEAD, then main/master/trunk/develop, then current branch).', + workflow: { + baseBranch: 'main', + }, + '// autoMerge': 'Only used when a goal opts in (coco goal start --auto-merge). User sensitiveGlobs/testGlobs are additive by default; set replaceDefault*Globs only when you intentionally replace defaults. coco.config.json is ALWAYS sensitive (self-tamper guard) regardless of this list.', autoMerge: DEFAULT_AUTO_MERGE_CONFIG, }; From 329a400574550cb4e2d747c74804a61586a00733 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:55:55 +0930 Subject: [PATCH 13/66] feat: require acknowledgement for verify policy changes --- src/commands/merge.ts | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/src/commands/merge.ts b/src/commands/merge.ts index b87ec45..94517e9 100644 --- a/src/commands/merge.ts +++ b/src/commands/merge.ts @@ -1,11 +1,22 @@ import { assessAutoMergeRisk, type RiskReport } from '../autoMergeRisk.js'; +import { VERIFY_TEST_COMMAND_CHANGE_ACK, verifyTestCommandChange } from '../cocoConfig.js'; import { checkout, ffMerge, gatherLive, headSha } from '../git.js'; import { mergeDecision } from '../gate.js'; import { improveOriginProtectedHits } from '../improve/originGate.js'; import { withLock } from '../lock.js'; import { findActiveGoal, touchAndWrite } from '../state.js'; -export function mergeGoal(repo: string, id: string): { merged: boolean; reason?: string } { +export interface MergeOptions { + ackVerifyPolicyChange?: boolean; +} + +function verifyPolicyMergeBlock(repo: string, base: string, ack?: boolean): string | undefined { + const change = verifyTestCommandChange(repo, base); + if (change === 'none' || ack) return undefined; + return `${VERIFY_TEST_COMMAND_CHANGE_ACK} (change: ${change})`; +} + +export function mergeGoal(repo: string, id: string, opts: MergeOptions = {}): { merged: boolean; reason?: string } { return withLock(repo, () => { const goal = findActiveGoal(repo); if (!goal || goal.id !== id) throw new Error(`coco: no active goal '${id}'`); @@ -21,7 +32,12 @@ export function mergeGoal(repo: string, id: string): { merged: boolean; reason?: return { merged: false, reason: `improve-origin change touches protected path(s): ${blocked.join(', ')} — a referee/metric change needs a human-authored goal, not a coco-improve one` }; } - // FF-only merge. ffMerge checks out `base` first; if the FF fails (main moved + // Verification policy changes are allowed only with an explicit human acknowledgement. This keeps + // the default human merge path from overlooking a goal that changed the very command coco trusted. + const policyBlock = verifyPolicyMergeBlock(repo, goal.base, opts.ackVerifyPolicyChange); + if (policyBlock) return { merged: false, reason: policyBlock }; + + // FF-only merge. ffMerge checks out `base` first; if the FF fails (base moved // between validation and here), restore the goal branch so a retry is sane. const res = ffMerge(repo, goal.base, goal.branch); if (!res.ok) { @@ -30,7 +46,7 @@ export function mergeGoal(repo: string, id: string): { merged: boolean; reason?: } goal.state = 'achieved'; - touchAndWrite(repo, goal, 'merge'); + touchAndWrite(repo, goal, opts.ackVerifyPolicyChange ? 'merge:verify-policy-ack' : 'merge'); return { merged: true }; }); } @@ -82,6 +98,11 @@ export function autoMergeGoal(repo: string, id: string, opts: { expectedSha: str const blocked = improveOriginProtectedHits(repo, goal); if (blocked.length) return toHuman(`improve-origin change touches protected path(s): ${blocked.join(', ')} — needs a human-authored referee-change goal, not a coco-improve one`); + // 3c. Auto-merge never acknowledges verify policy changes. Those must fall back to an explicit + // human merge command carrying --ack-verify-policy-change. + const policyBlock = verifyPolicyMergeBlock(repo, goal.base, false); + if (policyBlock) return toHuman(policyBlock); + // 4. Layer 2 risk-tier — policy read at base (tamper-resistant). A block means "let a human do it". const risk = assessAutoMergeRisk(repo, goal.base, goal.branch); if (!risk.allowed) return toHuman(risk.reason ?? 'auto-merge blocked by risk policy', risk); From cefed43b52bfe4b84e3f2dcd52ce3df7681c65de Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:56:29 +0930 Subject: [PATCH 14/66] feat: share package version discovery --- src/version.ts | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 src/version.ts diff --git a/src/version.ts b/src/version.ts new file mode 100644 index 0000000..abe2cc6 --- /dev/null +++ b/src/version.ts @@ -0,0 +1,25 @@ +import { existsSync, readFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +export function packageRoot(start = dirname(fileURLToPath(import.meta.url))): string | null { + let dir = start; + for (let i = 0; i < 8; i++) { + if (existsSync(join(dir, 'package.json'))) return dir; + const parent = dirname(dir); + if (parent === dir) break; + dir = parent; + } + return null; +} + +export function cocoVersion(): string { + try { + const root = packageRoot(); + if (!root) return 'unknown'; + const v = (JSON.parse(readFileSync(join(root, 'package.json'), 'utf8')) as { version?: string }).version; + return v || 'unknown'; + } catch { + return 'unknown'; + } +} From 792e3e5ae98cc529f7e52d1beb49cb86452c182b Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:57:12 +0930 Subject: [PATCH 15/66] feat: add codex setup assistant --- src/commands/setupCodex.ts | 83 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 src/commands/setupCodex.ts diff --git a/src/commands/setupCodex.ts b/src/commands/setupCodex.ts new file mode 100644 index 0000000..e5d5516 --- /dev/null +++ b/src/commands/setupCodex.ts @@ -0,0 +1,83 @@ +import { cpSync, existsSync, mkdirSync, readFileSync, readdirSync, statSync, writeFileSync } from 'node:fs'; +import { homedir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { packageRoot } from '../version.js'; + +export interface SetupCodexAction { + action: string; + target: string; + status: 'present' | 'would-write' | 'written' | 'would-copy' | 'copied' | 'missing-source' | 'skipped'; + detail?: string; +} + +export interface SetupCodexReport { + applied: boolean; + configPath: string; + skillsDir: string; + actions: SetupCodexAction[]; +} + +const COCO_MCP_BLOCK = `[mcp_servers.coco]\ncommand = "coco-mcp"\nargs = []\n`; +const ORACLE_HINT = `\n# Oracle is required for coco-loop plan/review gates. Configure your local Oracle MCP separately.\n# [mcp_servers.oracle]\n# command = "oracle"\n# args = ["mcp"]\n`; + +function uncommentedHasBlock(raw: string, block: string): boolean { + const uncommented = raw.split('\n').filter((l) => !l.trim().startsWith('#')).join('\n'); + return uncommented.includes(`[mcp_servers.${block}]`); +} + +function skillsSource(): string | null { + const root = packageRoot(); + if (!root) return null; + const src = join(root, 'skills'); + return existsSync(src) ? src : null; +} + +export function setupCodex(opts: { apply?: boolean; home?: string; configPath?: string; skillsDir?: string } = {}): SetupCodexReport { + const home = opts.home ?? homedir(); + const apply = opts.apply === true; + const configPath = resolve(opts.configPath ?? join(home, '.codex', 'config.toml')); + const skillsDir = resolve(opts.skillsDir ?? join(home, '.agents', 'skills')); + const actions: SetupCodexAction[] = []; + + const configExists = existsSync(configPath); + const configRaw = configExists ? readFileSync(configPath, 'utf8') : ''; + if (uncommentedHasBlock(configRaw, 'coco')) { + actions.push({ action: 'codex-mcp-config', target: configPath, status: 'present', detail: 'coco MCP block already configured' }); + } else { + const next = `${configRaw}${configRaw && !configRaw.endsWith('\n') ? '\n' : ''}\n${COCO_MCP_BLOCK}${uncommentedHasBlock(configRaw, 'oracle') ? '' : ORACLE_HINT}`; + if (apply) { + mkdirSync(dirname(configPath), { recursive: true }); + writeFileSync(configPath, next); + actions.push({ action: 'codex-mcp-config', target: configPath, status: 'written', detail: 'added coco MCP block' }); + } else { + actions.push({ action: 'codex-mcp-config', target: configPath, status: 'would-write', detail: 'would add coco MCP block' }); + } + } + + const src = skillsSource(); + if (!src) { + actions.push({ action: 'skills', target: skillsDir, status: 'missing-source', detail: 'package skills directory not found' }); + } else { + const names = readdirSync(src).filter((name) => { + try { + return statSync(join(src, name)).isDirectory(); + } catch { + return false; + } + }); + if (!names.length) actions.push({ action: 'skills', target: skillsDir, status: 'missing-source', detail: 'no skill directories found' }); + for (const name of names) { + const from = join(src, name); + const to = join(skillsDir, name); + if (apply) { + mkdirSync(skillsDir, { recursive: true }); + cpSync(from, to, { recursive: true, force: true }); + actions.push({ action: 'skill-sync', target: to, status: 'copied', detail: `synced ${name}` }); + } else { + actions.push({ action: 'skill-sync', target: to, status: existsSync(to) ? 'present' : 'would-copy', detail: `${name}${existsSync(to) ? ' already present' : ' would be copied'}` }); + } + } + } + + return { applied: apply, configPath, skillsDir, actions }; +} From 42044bf3f0cac0dd39cfc6a675ef202dd9a13d75 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:58:02 +0930 Subject: [PATCH 16/66] feat: add deterministic coco eval harness --- src/commands/eval.ts | 110 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 src/commands/eval.ts diff --git a/src/commands/eval.ts b/src/commands/eval.ts new file mode 100644 index 0000000..ffadad3 --- /dev/null +++ b/src/commands/eval.ts @@ -0,0 +1,110 @@ +import { deriveFacts } from '../epoch.js'; +import { FINGERPRINT_N } from '../fingerprint.js'; +import { nextAction } from '../gate.js'; +import { parseOracleVerdict } from '../oracleVerdict.js'; +import type { GoalEvent, GoalState } from '../types.js'; + +export interface EvalCaseResult { + id: string; + area: 'verdict' | 'epoch' | 'gate' | 'privacy' | 'self-improve'; + invariant: string; + ok: boolean; + detail?: string; +} + +export interface EvalReport { + ok: boolean; + total: number; + failed: number; + cases: EvalCaseResult[]; +} + +type EvalCase = Omit & { run: () => boolean; detail?: string }; + +const ev = (phase: GoalEvent['phase'], tree: string, verdict?: GoalEvent['verdict']): GoalEvent => ({ + phase, + tree, + verdict, + commit: `${tree}-commit`, + at: '2026-07-08T00:00:00.000Z', +}); + +function activeGoal(partial: Partial = {}): GoalState { + return { + id: 'eval-goal', + objective: 'eval', + branch: 'coco/eval-goal', + base: 'main', + baseTree: 'base-tree', + state: 'active', + maxFixRounds: 5, + acceptanceChecks: [], + events: [], + ...partial, + }; +} + +const cases: EvalCase[] = [ + { + id: 'oracle-verdict-ambiguous-fails-closed', + area: 'verdict', + invariant: 'Oracle text without an exact terminal verdict must not become clean.', + run: () => parseOracleVerdict('Looks good to me') === null, + }, + { + id: 'oracle-verdict-clean-is-strict', + area: 'verdict', + invariant: 'Only an exact VERDICT line grants a clean review verdict.', + run: () => parseOracleVerdict('Reviewed.\nVERDICT: clean') === 'clean', + }, + { + id: 'bad-verdict-sticks-to-tree', + area: 'epoch', + invariant: 'A tree that ever received a blocking review remains blocking for that tree.', + run: () => deriveFacts([ev('implement', 't1'), ev('review', 't1', 'blocking'), ev('review', 't1', 'clean')], 't1').latestReview === 'blocking', + }, + { + id: 'revert-to-old-clean-tree-requires-fresh-review', + area: 'epoch', + invariant: 'Returning to an old clean tree does not revive its approval after later work.', + run: () => deriveFacts([ev('implement', 'clean-tree'), ev('review', 'clean-tree', 'clean'), ev('implement', 'bad-tree'), ev('review', 'bad-tree', 'blocking')], 'clean-tree').latestReview === 'none', + }, + { + id: 'fingerprint-loop-escalates', + area: 'gate', + invariant: 'The same repeated failure fingerprint escalates to a human instead of looping forever.', + run: () => + nextAction( + activeGoal({ + events: [ev('implement', 't1'), ev('review', 't1', 'blocking')], + failureLoop: { key: 'same-failure', count: FINGERPRINT_N, history: [] }, + }), + { tHead: 't1', treeClean: true, onBranch: true, baseMerged: true }, + ) === 'escalate-human', + }, + { + id: 'review-unavailable-pauses-above-git-recovery', + area: 'gate', + invariant: 'Oracle unavailable/ambiguous verdict pauses durably before normal git recovery advice.', + run: () => + nextAction( + activeGoal({ + reviewUnavailable: { at: '2026-07-08T00:00:00.000Z', phase: 'review', commit: 'c', tree: 't', reason: 'ambiguous-verdict', attempts: 1 }, + }), + { tHead: 't', treeClean: false, onBranch: true, baseMerged: true }, + ) === 'escalate-human', + }, +]; + +export function runEval(): EvalReport { + const results = cases.map((c): EvalCaseResult => { + try { + const ok = c.run(); + return { id: c.id, area: c.area, invariant: c.invariant, ok, ...(ok ? {} : { detail: c.detail ?? 'invariant returned false' }) }; + } catch (e) { + return { id: c.id, area: c.area, invariant: c.invariant, ok: false, detail: (e as Error).message }; + } + }); + const failed = results.filter((r) => !r.ok).length; + return { ok: failed === 0, total: results.length, failed, cases: results }; +} From 730cee67e9e62fa8dbda913041f6d56d1fc243f7 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:58:31 +0930 Subject: [PATCH 17/66] feat: add command descriptor registry --- src/commands/registry.ts | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 src/commands/registry.ts diff --git a/src/commands/registry.ts b/src/commands/registry.ts new file mode 100644 index 0000000..1a422e8 --- /dev/null +++ b/src/commands/registry.ts @@ -0,0 +1,32 @@ +export type CommandEffect = 'read' | 'write' | 'destructive' | 'external'; + +export interface CommandDescriptor { + name: string; + surfaces: ('cli' | 'mcp' | 'skill')[]; + effect: CommandEffect; + summary: string; + safety?: string; +} + +const COMMANDS: readonly CommandDescriptor[] = [ + { name: 'coco init', surfaces: ['cli', 'mcp'], effect: 'write', summary: 'Bootstrap .coco and tracked coco.config.json on a clean repo.' }, + { name: 'coco goal start', surfaces: ['cli', 'mcp', 'skill'], effect: 'write', summary: 'Create one active goal branch from the configured base branch.', safety: 'Refuses dirty trees and concurrent active goals.' }, + { name: 'coco goal status', surfaces: ['cli', 'mcp', 'skill'], effect: 'read', summary: 'Derive deterministic nextAction from goal state and live git.' }, + { name: 'coco goal record', surfaces: ['cli', 'mcp', 'skill'], effect: 'write', summary: 'Append plan/implement/review events bound to expectedSha.', safety: 'Review verdicts are parsed from Oracle output; verify is not accepted here.' }, + { name: 'coco goal verify', surfaces: ['cli', 'mcp', 'skill'], effect: 'external', summary: 'Run the committed verify.testCommand and record pass/fail from exit code.', safety: 'Coco owns verify; agents cannot self-report pass.' }, + { name: 'coco merge', surfaces: ['cli', 'skill'], effect: 'destructive', summary: 'Human-terminal FF-only merge path.', safety: 'Requires clean review, passing verify, current epoch, clean branch, base ancestry, and explicit ack for verify policy changes.' }, + { name: 'coco_merge', surfaces: ['mcp', 'skill'], effect: 'destructive', summary: 'Opt-in auto-merge attempt for a single goal.', safety: 'Requires per-goal consent and risk-tier; falls back to human on risk.' }, + { name: 'coco health', surfaces: ['cli', 'mcp'], effect: 'read', summary: 'Report loop health, stall, conflict, in-flight, and invalid-state conditions.' }, + { name: 'coco doctor', surfaces: ['cli'], effect: 'read', summary: 'Aggregate local environment, repo, wiring, goal, and data hygiene checks.' }, + { name: 'coco doctor clean', surfaces: ['cli'], effect: 'destructive', summary: 'Dry-run/apply cleanup of terminal or orphaned verify-run cache only.' }, + { name: 'coco improve digest', surfaces: ['cli', 'skill'], effect: 'read', summary: 'Summarise structural audit signals with an insufficient-data gate.' }, + { name: 'coco improve check', surfaces: ['cli', 'skill'], effect: 'read', summary: 'Refuse protected-path targets for self-improvement proposals.' }, + { name: 'coco improve promote', surfaces: ['cli', 'skill'], effect: 'write', summary: 'Promote one non-protected improve task linked to a local improve-spec.' }, + { name: 'coco eval', surfaces: ['cli'], effect: 'read', summary: 'Run deterministic safety-regression fixture checks.' }, + { name: 'coco setup codex', surfaces: ['cli'], effect: 'write', summary: 'Dry-run/apply Codex MCP config and skill sync.' }, + { name: 'coco-store', surfaces: ['cli', 'skill'], effect: 'write', summary: 'PM layer for ResourceCards, roadmap, backlog, briefs, and visualisation.', safety: 'Must not mutate .coco/goals.' }, +]; + +export function listCommandDescriptors(): CommandDescriptor[] { + return COMMANDS.map((c) => ({ ...c, surfaces: [...c.surfaces] })); +} From 0663290c4f96987c08f0df199b106e5c961eb7fe Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:00:58 +0930 Subject: [PATCH 18/66] feat: wire setup eval registry and safer merge CLI --- src/cli.ts | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/src/cli.ts b/src/cli.ts index 5bf6b2e..47f78bb 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -38,6 +38,9 @@ import { cocoDone, cocoNext } from './commands/backlog.js'; import { auditReport } from './commands/audit.js'; import { readAudit } from './audit.js'; import { cleanDoctor, runDoctor } from './commands/doctor.js'; +import { runEval } from './commands/eval.js'; +import { listCommandDescriptors } from './commands/registry.js'; +import { setupCodex } from './commands/setupCodex.js'; import { improveDigest } from './improve/digest.js'; import { improvePromote } from './improve/promote.js'; import { improveCheck, improveCheckDiff } from './improve/protected.js'; @@ -61,10 +64,37 @@ export function main(argv: string[] = process.argv.slice(2)): number { return 0; } + if (cmd === 'commands') { + out({ commands: listCommandDescriptors() }); + return 0; + } + + if (cmd === 'eval') { + const res = runEval(); + out(res); + return res.ok ? 0 : 1; + } + + if (cmd === 'setup') { + if (sub === 'codex') { + const { values } = parseArgs({ + args: rest, + options: { apply: { type: 'boolean' }, home: { type: 'string' }, config: { type: 'string' }, 'skills-dir': { type: 'string' } }, + }); + out(setupCodex({ apply: values.apply === true, home: values.home, configPath: values.config, skillsDir: values['skills-dir'] })); + return 0; + } + // fall through to unknown-command for typo'd setup subcommands + } + if (cmd === 'merge') { - const { values } = parseArgs({ args: [sub, ...rest].filter(Boolean), options: { goal: { type: 'string' } }, allowPositionals: true }); + const { values } = parseArgs({ + args: [sub, ...rest].filter(Boolean), + options: { goal: { type: 'string' }, 'ack-verify-policy-change': { type: 'boolean' } }, + allowPositionals: true, + }); if (!values.goal) throw new Error('coco merge requires --goal'); - out(mergeGoal(repo, values.goal)); + out(mergeGoal(repo, values.goal, { ackVerifyPolicyChange: values['ack-verify-policy-change'] === true })); return 0; } @@ -192,6 +222,7 @@ export function main(argv: string[] = process.argv.slice(2)): number { options: { objective: { type: 'string' }, acceptance: { type: 'string' }, + base: { type: 'string' }, 'max-fix-rounds': { type: 'string' }, 'max-wall-min': { type: 'string' }, 'auto-merge': { type: 'boolean' }, @@ -203,6 +234,7 @@ export function main(argv: string[] = process.argv.slice(2)): number { objective: values.objective, acceptanceChecks: values.acceptance ? values.acceptance.split(';').map((s) => s.trim()).filter(Boolean) : [], maxFixRounds: values['max-fix-rounds'] ? Number(values['max-fix-rounds']) : 5, + base: values.base, autoMergeAllowed: values['auto-merge'] === true ? true : undefined, budget: values['max-wall-min'] ? { maxWallClockMin: Number(values['max-wall-min']) } : undefined, }), From 56700f8297667825bcaaae075f4d4b6d5640cc87 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:02:03 +0930 Subject: [PATCH 19/66] feat: expose base branch override through MCP goal start --- src/mcp/tools.ts | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/mcp/tools.ts b/src/mcp/tools.ts index 28071d3..bbae139 100644 --- a/src/mcp/tools.ts +++ b/src/mcp/tools.ts @@ -27,6 +27,7 @@ export function cocoGoalStart(a: { acceptanceChecks?: string[]; maxFixRounds?: number; backlogTaskId?: string; + base?: string; autoMergeAllowed?: boolean; budget?: { maxWallClockMin?: number }; }): { goalId: string; status: StatusReport } { @@ -36,6 +37,7 @@ export function cocoGoalStart(a: { acceptanceChecks: a.acceptanceChecks ?? [], maxFixRounds: a.maxFixRounds ?? 5, backlogTaskId: a.backlogTaskId, + base: a.base, autoMergeAllowed: a.autoMergeAllowed, budget: a.budget, }); From 989400e6ca60e0013d30c27ad747606d7599fc34 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:03:39 +0930 Subject: [PATCH 20/66] feat: sync MCP server version and base schema --- src/mcp/server.ts | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/mcp/server.ts b/src/mcp/server.ts index 67fc231..fcacd6f 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -1,5 +1,6 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; import { z } from 'zod'; +import { cocoVersion } from '../version.js'; import { cocoDoneTool, cocoGoalClear, @@ -42,7 +43,7 @@ function wrap(fn: (a: T) => unknown): (a: T) => Promise { } export function buildServer(): McpServer { - const server = new McpServer({ name: 'coco', version: '0.0.1' }); + const server = new McpServer({ name: 'coco', version: cocoVersion() }); const repoDir = z.string().describe('Absolute path to the target git repository (the project cwd).'); server.registerTool( @@ -59,13 +60,14 @@ export function buildServer(): McpServer { 'coco_goal_start', { title: 'Start a coco goal', - description: 'Create a goal + coco/ branch. Refuses if a goal is already active or the tree is dirty. Returns goalId + status.', + description: 'Create a goal + coco/ branch from workflow.baseBranch / repo default / explicit base. Refuses if a goal is already active or the tree is dirty. Returns goalId + status.', inputSchema: { repoDir, objective: z.string().min(1), acceptanceChecks: z.array(z.string()).optional(), maxFixRounds: z.number().int().positive().optional(), backlogTaskId: z.string().optional().describe('the BACKLOG.md task id this goal implements (from coco_next), so coco_done survives a session drop'), + base: z.string().optional().describe('optional per-goal base branch override; defaults to workflow.baseBranch / repo default branch'), autoMergeAllowed: z.boolean().optional().describe('Layer 2 forward consent: allow coco_merge to auto-merge THIS goal if it comes back clean + verified + rebased + within risk policy. Opt-in per goal only (from `$coco-loop --auto`); never a global default.'), budget: z.object({ maxWallClockMin: z.number().positive() }).optional().describe('optional wall-clock cap (minutes); over budget mid-loop → coco_health returns `budget-exceeded`'), }, @@ -213,7 +215,7 @@ export function buildServer(): McpServer { { title: 'Auto-merge an opted-in goal (Layer 2)', description: - 'Merge a goal WITHOUT a human click — allowed ONLY when the goal opted in at start (autoMergeAllowed) AND it passes every merge gate (review clean, verify pass, rebased) AND the risk-tier (no sensitive paths, within size limit, has tests). expectedSha must equal current HEAD. Returns {merged:true, mergedSha} on success, else {merged:false, reason, next}: next="human-merge" → fall back to proposing `coco merge --goal ` for a human; next="continue-loop" → a transient gate (rebase/re-review), re-run coco_goal_status. A non-consented, non-green, or risky goal can NEVER merge here — the human `coco merge` CLI is the manual path.', + 'Merge a goal WITHOUT a human click — allowed ONLY when the goal opted in at start (autoMergeAllowed) AND it passes every merge gate (review clean, verify pass, rebased) AND the risk-tier (no sensitive paths, within size limit, has tests). expectedSha must equal current HEAD. Returns {merged:true, mergedSha} on success, else {merged:false, reason, next}: next="human-merge" → fall back to proposing `coco merge --goal ` for a human; next="continue-loop" → a transient gate (rebase/re-review), re-run coco_goal_status. A non-consented, non-green, risky, or verify-policy-changing goal can NEVER merge here — the human `coco merge --goal --ack-verify-policy-change` CLI remains the manual acknowledgement path when verify.testCommand changed.', inputSchema: { repoDir, goalId: z.string(), expectedSha: z.string().describe('must equal current HEAD (the latest coco_goal_status headSha)') }, }, wrap(cocoMerge), @@ -221,7 +223,7 @@ export function buildServer(): McpServer { // NOTE: coco_merge auto-merges ONLY a goal that opted in at start AND passes every // mergeDecision gate AND the risk-tier (all enforced server-side in autoMergeGoal). - // A confused/runaway loop still cannot merge a non-consented, non-green, or risky - // goal. The human `coco merge --goal ` CLI remains the manual consent path. + // A confused/runaway loop still cannot merge a non-consented, non-green, risky, or + // verify-policy-changing goal. The human `coco merge --goal ` CLI remains the manual consent path. return server; } From 2f53f1e2da85b57f8308469702b6d4fdcc544c6c Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:04:12 +0930 Subject: [PATCH 21/66] chore: include deterministic eval in ci --- package.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/package.json b/package.json index fa70c59..1ee4c45 100644 --- a/package.json +++ b/package.json @@ -39,7 +39,8 @@ "build": "tsdown", "typecheck": "tsc --noEmit", "test": "vitest run", - "ci": "pnpm typecheck && pnpm test && pnpm build", + "eval": "tsx src/bin/coco.ts eval", + "ci": "pnpm typecheck && pnpm test && pnpm eval && pnpm build", "dev": "tsx src/bin/coco.ts", "prepublishOnly": "pnpm ci" }, From ac2318606b8a1b6806c50666fa80980a6b4d2369 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:04:29 +0930 Subject: [PATCH 22/66] ci: run unified pnpm ci gate --- .github/workflows/ci.yml | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 84210c2..730bb3e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,11 +34,5 @@ jobs: - name: Install dependencies run: pnpm install --frozen-lockfile - - name: Typecheck - run: pnpm typecheck - - - name: Test - run: pnpm test - - - name: Build - run: pnpm build + - name: Verify + run: pnpm ci From 5ac20d2cd941fb19d774e614d36fe292e6cb0373 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:05:38 +0930 Subject: [PATCH 23/66] test: cover goal schema validation --- tests/state.test.ts | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/tests/state.test.ts b/tests/state.test.ts index fb57cb7..28d181c 100644 --- a/tests/state.test.ts +++ b/tests/state.test.ts @@ -1,5 +1,6 @@ -import { existsSync, mkdirSync } from 'node:fs'; +import { existsSync, mkdirSync, writeFileSync } from 'node:fs'; import { expect, test } from 'vitest'; +import { GOAL_SCHEMA_VERSION } from '../src/goalSchema.js'; import { withLock } from '../src/lock.js'; import { cocoDir } from '../src/paths.js'; import { findActiveGoal, goalPath, readGoal, writeGoal } from '../src/state.js'; @@ -10,12 +11,22 @@ function newGoal(id: string, state: GoalState['state'] = 'active'): GoalState { return { id, objective: 'x', branch: `coco/${id}`, base: 'main', state, maxFixRounds: 3, acceptanceChecks: [], events: [] }; } -test('writeGoal + readGoal round-trips atomically', () => { +test('writeGoal + readGoal round-trips atomically and stamps schemaVersion', () => { const repo = tmpRepo(); mkdirSync(cocoDir(repo) + '/goals', { recursive: true }); const p = goalPath(repo, 'g1'); writeGoal(p, newGoal('g1')); - expect(readGoal(p).id).toBe('g1'); + const got = readGoal(p); + expect(got.id).toBe('g1'); + expect(got.schemaVersion).toBe(GOAL_SCHEMA_VERSION); +}); + +test('readGoal rejects malformed-but-parseable goal state', () => { + const repo = tmpRepo(); + mkdirSync(cocoDir(repo) + '/goals', { recursive: true }); + const p = goalPath(repo, 'bad'); + writeFileSync(p, JSON.stringify({ id: 'bad', objective: 'x', branch: 'coco/bad', base: 'main', state: 'active', maxFixRounds: 3, acceptanceChecks: [] })); + expect(() => readGoal(p)).toThrow(/schema invalid|events/); }); test('findActiveGoal returns only the active goal', () => { From b81372ad081f477e7d202f07e3a6915a8ff4191a Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:06:12 +0930 Subject: [PATCH 24/66] test: cover goal base branch and id collision --- tests/goalStart.test.ts | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/tests/goalStart.test.ts b/tests/goalStart.test.ts index 6624417..a19d1e6 100644 --- a/tests/goalStart.test.ts +++ b/tests/goalStart.test.ts @@ -1,10 +1,11 @@ -import { existsSync } from 'node:fs'; +import { existsSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; import { expect, test } from 'vitest'; import { initRepo } from '../src/commands/init.js'; import { goalStart } from '../src/commands/goalStart.js'; import { goalPath, readGoal } from '../src/state.js'; import { currentBranch } from '../src/git.js'; -import { tmpRepo } from './helpers.js'; +import { commit, g, tmpRepo } from './helpers.js'; test('goal start creates a goal file, branch, and returns nextAction=plan', () => { const repo = tmpRepo(); @@ -16,6 +17,31 @@ test('goal start creates a goal file, branch, and returns nextAction=plan', () = expect(currentBranch(repo)).toBe(`coco/${res.goalId}`); }); +test('goal start uses an explicit base branch override', () => { + const repo = tmpRepo(); + initRepo(repo); + g(repo, ['checkout', '-b', 'develop']); + commit(repo, 'dev.txt', 'dev\n', 'develop work'); + g(repo, ['checkout', 'main']); + const id = goalStart(repo, { objective: 'from develop', maxFixRounds: 5, acceptanceChecks: [], base: 'develop' }).goalId; + expect(readGoal(goalPath(repo, id)).base).toBe('develop'); + expect(currentBranch(repo)).toBe(`coco/${id}`); +}); + +test('goal start allocates a suffix when the minute/objective id already exists', () => { + const repo = tmpRepo(); + initRepo(repo); + const now = new Date('2026-07-08T01:02:03.000Z'); + const first = goalStart(repo, { objective: 'same', maxFixRounds: 5, acceptanceChecks: [], now }).goalId; + // Simulate a terminal prior goal so a second start is allowed while its branch/file remain. + const p = goalPath(repo, first); + const g1 = readGoal(p); + writeFileSync(p, JSON.stringify({ ...g1, state: 'cancelled' }, null, 2)); + g(repo, ['checkout', 'main']); + const second = goalStart(repo, { objective: 'same', maxFixRounds: 5, acceptanceChecks: [], now }).goalId; + expect(second).toBe(`${first}-2`); +}); + test('goal start refuses if the repo was not coco-initialized (no `coco init`)', () => { const repo = tmpRepo(); // a git repo, but no coco init (no .gitignore .coco/) expect(() => goalStart(repo, { objective: 'x', maxFixRounds: 5, acceptanceChecks: [] })).toThrow(/coco init/); From 9484ed0f57bf33c589ff0d6083eef3a128c5e3b6 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:07:00 +0930 Subject: [PATCH 25/66] test: cover workflow and additive auto-merge config --- tests/cocoConfig.test.ts | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/tests/cocoConfig.test.ts b/tests/cocoConfig.test.ts index fea84a6..f3923f8 100644 --- a/tests/cocoConfig.test.ts +++ b/tests/cocoConfig.test.ts @@ -4,6 +4,7 @@ import { expect, test } from 'vitest'; import { DEFAULT_AUTO_MERGE_CONFIG, readAutoMergePolicyAtRef, + resolveBaseBranch, VERIFY_NOT_CONFIGURED_WARNING, verifyConfigWarnings, } from '../src/cocoConfig.js'; @@ -32,6 +33,12 @@ function commitConfig(repo: string, body: unknown): void { g(repo, ['commit', '-m', 'config']); } +test('resolveBaseBranch honours workflow.baseBranch', () => { + const repo = tmpRepo(); + commitConfig(repo, { workflow: { baseBranch: 'develop' } }); + expect(resolveBaseBranch(repo)).toBe('develop'); +}); + test('readAutoMergePolicyAtRef returns defaults when config is missing', () => { const repo = tmpRepo(); // no coco.config.json expect(readAutoMergePolicyAtRef(repo, 'HEAD')).toEqual(DEFAULT_AUTO_MERGE_CONFIG); @@ -46,6 +53,22 @@ test('readAutoMergePolicyAtRef merges a partial autoMerge block over defaults', expect(p.testGlobs).toEqual(DEFAULT_AUTO_MERGE_CONFIG.testGlobs); }); +test('user autoMerge globs are additive by default', () => { + const repo = tmpRepo(); + commitConfig(repo, { autoMerge: { sensitiveGlobs: ['infra/**'], testGlobs: ['checks/**'] } }); + const p = readAutoMergePolicyAtRef(repo, 'HEAD'); + expect(p.sensitiveGlobs).toEqual([...DEFAULT_AUTO_MERGE_CONFIG.sensitiveGlobs, 'infra/**']); + expect(p.testGlobs).toEqual([...DEFAULT_AUTO_MERGE_CONFIG.testGlobs, 'checks/**']); +}); + +test('autoMerge globs can explicitly replace defaults', () => { + const repo = tmpRepo(); + commitConfig(repo, { autoMerge: { replaceDefaultSensitiveGlobs: true, sensitiveGlobs: ['infra/**'], replaceDefaultTestGlobs: true, testGlobs: ['checks/**'] } }); + const p = readAutoMergePolicyAtRef(repo, 'HEAD'); + expect(p.sensitiveGlobs).toEqual(['infra/**']); + expect(p.testGlobs).toEqual(['checks/**']); +}); + test('readAutoMergePolicyAtRef falls back to defaults on malformed JSON or bad types', () => { const bad = tmpRepo(); writeFileSync(join(bad, 'coco.config.json'), '{ not json'); From 71c24bcdc671ed46921ae3796abd99d8862a9f84 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:08:22 +0930 Subject: [PATCH 26/66] test: require ack for verify policy merge changes --- tests/merge.test.ts | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tests/merge.test.ts b/tests/merge.test.ts index 1aba954..0e79b13 100644 --- a/tests/merge.test.ts +++ b/tests/merge.test.ts @@ -93,6 +93,28 @@ test('goal clear cancels the active goal', () => { expect(readGoal(goalPath(repo, id)).state).toBe('cancelled'); }); +test('human merge requires explicit acknowledgement when verify.testCommand changed', () => { + const repo = tmpRepo(); + initRepo(repo); + writeFileSync(join(repo, 'coco.config.json'), JSON.stringify({ verify: { testCommand: 'pnpm test' } })); + g(repo, ['add', 'coco.config.json']); + g(repo, ['commit', '-m', 'base verify command']); + const id = goalStart(repo, { objective: 'policy change', maxFixRounds: 3, acceptanceChecks: [] }).goalId; + goalRecord(repo, { goal: id, phase: 'plan', expectedSha: headSha(repo) }); + writeFileSync(join(repo, 'f.txt'), 'x\n'); + writeFileSync(join(repo, 'coco.config.json'), JSON.stringify({ verify: { testCommand: 'true' } })); + g(repo, ['add', '-A']); + g(repo, ['commit', '-m', 'work and verify policy change']); + goalRecord(repo, { goal: id, phase: 'implement', expectedSha: headSha(repo) }); + goalRecord(repo, { goal: id, phase: 'review', verdict: 'clean', expectedSha: headSha(repo) }); + goalRecord(repo, { goal: id, phase: 'verify', verdict: 'pass', expectedSha: headSha(repo) }); + + const blocked = mergeGoal(repo, id); + expect(blocked.merged).toBe(false); + expect(blocked.reason).toMatch(/ack-verify-policy-change/); + expect(mergeGoal(repo, id, { ackVerifyPolicyChange: true }).merged).toBe(true); +}); + // --- Layer 2 auto-merge --- test('autoMergeGoal fast-forwards and marks achieved on a green, low-risk diff with consent', () => { From e4da2007979cd42993bd910ef78b8b165614e659 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:09:32 +0930 Subject: [PATCH 27/66] fix: surface ack merge command on auto policy fallback --- src/commands/merge.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/commands/merge.ts b/src/commands/merge.ts index 94517e9..c2571b6 100644 --- a/src/commands/merge.ts +++ b/src/commands/merge.ts @@ -101,7 +101,7 @@ export function autoMergeGoal(repo: string, id: string, opts: { expectedSha: str // 3c. Auto-merge never acknowledges verify policy changes. Those must fall back to an explicit // human merge command carrying --ack-verify-policy-change. const policyBlock = verifyPolicyMergeBlock(repo, goal.base, false); - if (policyBlock) return toHuman(policyBlock); + if (policyBlock) return { ...toHuman(policyBlock), humanCommand: `coco merge --goal ${id} --ack-verify-policy-change` }; // 4. Layer 2 risk-tier — policy read at base (tamper-resistant). A block means "let a human do it". const risk = assessAutoMergeRisk(repo, goal.base, goal.branch); From 291bfa1426700c16a959a2cd79d721bf25b46032 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:10:25 +0930 Subject: [PATCH 28/66] test: avoid verify policy change in happy path --- tests/e2e.test.ts | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/tests/e2e.test.ts b/tests/e2e.test.ts index d184e7b..6886127 100644 --- a/tests/e2e.test.ts +++ b/tests/e2e.test.ts @@ -24,14 +24,18 @@ function head(repo: string): string { test('end-to-end happy path via the CLI reaches merge-gate then merges', { timeout: 30000 }, () => { const repo = mkdtempSync(join(tmpdir(), 'coco-e2e-')); coco(repo, ['init']); + // Configure verify on the base branch before the goal starts. Changing verify.testCommand inside + // a goal is now a separate human-acknowledged policy change, not the happy path. + writeFileSync(join(repo, 'coco.config.json'), `${JSON.stringify({ verify: { testCommand: 'true' } })}\n`); + gitc(repo, ['add', 'coco.config.json']); + gitc(repo, ['commit', '-m', 'configure verify']); const id = JSON.parse(coco(repo, ['goal', 'start', '--objective', 'e2e feature'])).goalId; coco(repo, ['goal', 'record', '--goal', id, '--phase', 'plan', '--expected-sha', head(repo)]); - // a real change + the tracked verify config so coco runs the tests itself (exit 0 → pass) + // a real change; coco runs the base-configured tests itself (exit 0 → pass) writeFileSync(join(repo, 'f.txt'), 'feature\n'); - writeFileSync(join(repo, 'coco.config.json'), `${JSON.stringify({ verify: { testCommand: 'true' } })}\n`); - gitc(repo, ['add', 'f.txt', 'coco.config.json']); + gitc(repo, ['add', 'f.txt']); gitc(repo, ['commit', '-m', 'impl']); coco(repo, ['goal', 'record', '--goal', id, '--phase', 'implement', '--expected-sha', head(repo)]); From f0665ddb7c4cec5750911c058e0b75d6d86890c1 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:12:49 +0930 Subject: [PATCH 29/66] docs: update loop skill for base branch and verify policy ack --- skills/coco-loop/SKILL.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/skills/coco-loop/SKILL.md b/skills/coco-loop/SKILL.md index bc5447a..eaee4ef 100644 --- a/skills/coco-loop/SKILL.md +++ b/skills/coco-loop/SKILL.md @@ -16,7 +16,7 @@ Invoke: `$coco-loop ` (Codex) / `/coco-loop ` (Claude Code 3. **Chain `expectedSha`** for `coco_goal_record` from the `headSha` in the most recent `coco_goal_status` (or the status returned by the previous record). 4. **Commit before review/verify.** The referee refuses review/verify on a dirty tree. 5. **Oracle verdict is strict.** Your review consult MUST end with a line exactly `VERDICT: clean` or `VERDICT: blocking`. Pass Oracle's raw output as `reviewOutput` — coco parses it. If Oracle is unreachable, errors, times out, or the verdict is missing/ambiguous: **retry ONCE; if it still fails, call `coco_goal_oracle_unavailable({ goalId, phase:"plan"|"review", reason, attempts })`, then STOP and tell the user.** Never invent a verdict. coco marks the goal `review-unavailable` → `escalate-human` and refuses merge, so the loop pauses durably (never false-green). After the user resolves Oracle (re-login / restart), `coco_goal_op_clear({ goalId })` to resume. -6. **You never merge without consent.** At `merge-gate`, if the goal opted in (`autoMergeAllowed`, from `$coco-loop --auto`) call `coco_merge` — it merges ONLY when green + rebased + within risk policy, else it returns you to the human path. Otherwise (or on a `human-merge` fallback) present the merge card and **propose** the exact `coco merge --goal ` command as an executable step so the harness's approval prompt gates it. Never bundle a merge silently, never request an always-allow, and never proceed past a declined or unexecuted merge. +6. **You never merge without consent.** At `merge-gate`, if the goal opted in (`autoMergeAllowed`, from `$coco-loop --auto`) call `coco_merge` — it merges ONLY when green + rebased + within risk policy, else it returns you to the human path. Otherwise (or on a `human-merge` fallback) present the merge card and **propose the exact `humanCommand` returned by coco, or `coco merge --goal ` if no command was returned, as an executable step** so the harness's approval prompt gates it. If verify policy changed, the human command must include `--ack-verify-policy-change`. Never bundle a merge silently, never request an always-allow, and never proceed past a declined or unexecuted merge. 7. **Wrap long Oracle ops.** Right before a long Oracle consult (plan/review), call `coco_goal_op_start({ repoDir, goalId, phase, kind:"oracle" })` so `coco_health` reads `operation-in-progress` instead of misreading the pause as `stalled`. The matching `coco_goal_record` clears it (or `coco_goal_op_clear` if you abort). (**verify is coco-owned** — `coco_goal_verify_start` handles its own in-flight marker; don't op-start it.) An op still in flight after 1h → `in-flight-timeout`. 8. **Run Oracle review consults in a subagent.** A browser `consult` returns a large payload (minutes of `[browser] Waiting…` polling breadcrumbs + the full transcript) that rots the driver context. Dispatch the review consult to a general-purpose subagent whose whole job is: run the `consult`, then return ONLY `{ verdict, findings[], verdictBlock }` — `verdictBlock` is the answer text through the final `VERDICT:` line (pass it verbatim as `reviewOutput` so coco parses server-side), `findings[]` is one line each. The polling noise and the ~80 KB transcript stay in the subagent; coco persists verdict+findings as event evidence, so the driver keeps everything that matters and none of the noise. (The raw transcript stays at `~/.oracle/sessions//artifacts/transcript.md` — never `Read` it into the driver.) 9. **Echo the progress card on transitions (so the human sees progress natively).** `coco_goal_status` returns `progress.markdown` — a ready-to-render fenced checkpoint card. Echo it **verbatim, as its own message**, whenever the situation *changes*: `nextAction` differs from the previous cycle, a review returns clean/blocking, a verify starts or finishes, a blocker appears, or `merge-gate` is reached. **Do NOT re-echo an unchanged card every poll** — on an unchanged `nextAction`, act silently (repeated near-identical cards just add scroll + context noise). The card is a read-only snapshot; keep the merge command (rule 6) **outside** it as the separate, unambiguous approval step. This is the only native-progress surface in the Codex macOS app — assistant markdown can't update in place, so each echo is a fresh checkpoint, not a live bar. @@ -25,7 +25,7 @@ Invoke: `$coco-loop ` (Codex) / `/coco-loop ` (Claude Code 1. `coco_init({ repoDir })`. 2. **Objective:** if the user gave one, use it. If not, call `coco_next({ repoDir })` — if it returns a task, use its `title` + `body` as the objective and remember its `id` (the backlog task); if it returns `null`, tell the user the backlog has nothing ready and stop. -3. `coco_goal_start({ repoDir, objective, acceptanceChecks?, maxFixRounds?, backlogTaskId?, autoMergeAllowed? })` → `goalId`. If step 2 came from the backlog, pass its `id` as `backlogTaskId` so it's stored on the goal (durable across session drops). Pass `autoMergeAllowed: true` ONLY if the user invoked with `--auto` (forward consent for Layer 2 auto-merge). +3. `coco_goal_start({ repoDir, objective, acceptanceChecks?, maxFixRounds?, backlogTaskId?, base?, autoMergeAllowed? })` → `goalId`. If step 2 came from the backlog, pass its `id` as `backlogTaskId` so it's stored on the goal (durable across session drops). Omit `base` unless the user explicitly wants a non-default base; coco otherwise uses `workflow.baseBranch` / repo default. Pass `autoMergeAllowed: true` ONLY if the user invoked with `--auto` (forward consent for Layer 2 auto-merge). 4. **Pack the context brief + ground with background (best-effort).** Right after `goalId`, assemble the brief the `plan` consult will read: - **Background — default is the current context.** Unless the user gave a file, distill the **current conversation/session** — the intent, decisions, constraints, and relevant file paths you just established with the user — into a short structured block (**Goal / Approach / Constraints / Relevant files**) and pipe it through the **same bounder** so it's capped, not pasted unbounded: `printf '%s' "" | coco-store pack --goal --query "" --background-stdin`. This grounds the loop in what was just discussed instead of planning blind. Keep it to what you actually established with the user; don't paste raw file dumps or secrets. - **Background — from a file.** If the user pointed you at a doc, pass it through the deterministic bounder: `coco-store pack --goal --query "" --background `. pack requires the file to resolve **inside the repo** (it refuses outside/symlinked/secret-looking/binary files — the brief is sent to Oracle), head-bounds it (~150 lines / 6 KB), labels its freshness (`untracked` / `uncommitted local edits` / `[STALE?]`), and places it **first** in the brief. @@ -35,23 +35,27 @@ Invoke: `$coco-loop ` (Codex) / `/coco-loop ` (Claude Code - **`plan`** → `coco_goal_op_start({ phase:"plan", kind:"oracle" })`, then `consult` (Oracle, deep/plan mode, `engine:"browser"`) with the relevant files **and the coco-store brief from step 4 (if any)** → capture the plan. `coco_goal_record({ repoDir, goalId, phase:"plan", expectedSha:, evidence:"" })`. - **`implement`** / **`fix`** → make the edits, run the local tests, **commit**. `coco_goal_record({ phase:"implement", expectedSha:, evidence:"" })`. - - **`review`** → `coco_goal_op_start({ phase:"review", kind:"oracle" })`, then run the review **via a subagent** (rule 8): `consult` (Oracle, review mode, `engine:"browser"`) on the committed diff. **Ask for the whole failure surface in ONE pass, not one bug per round:** for state-machine / filesystem / parser / git-touching code, instruct Oracle to *enumerate every branch of the changed function(s) and give a concrete failing scenario per branch* (e.g. unborn repo, existing repo, ignored file, staged/removed file, idempotent re-run), then finish with a single `VERDICT: clean` or `VERDICT: blocking`. This turns serial N-round bug discovery into 1–2 rounds — fewer consults, less chance of tripping ChatGPT's rate limit. `coco_goal_record({ phase:"review", expectedSha:, evidence:"<1–2 sentence blocking reason>", reviewOutput:"" })`. If unavailable/ambiguous → STOP (rule 5). + - **`review`** → `coco_goal_op_start({ phase:"review", kind:"oracle" })`, then run the review **via a subagent** (rule 8): `consult` (Oracle, review mode, `engine:"browser"`) on the committed diff. **Ask for the whole failure surface in ONE pass, not one bug per round:** for state-machine / filesystem / parser / git-touching code, instruct Oracle to *enumerate every branch of the changed function(s) and give a concrete failing scenario per branch* (e.g. unborn repo, existing repo, ignored file, staged/removed file, idempotent re-run), then finish with a single `VERDICT: clean` or `VERDICT: blocking`. `coco_goal_record({ phase:"review", expectedSha:, evidence:"<1–2 sentence blocking reason>", reviewOutput:"" })`. If unavailable/ambiguous → STOP (rule 5). - **`verify`** → **coco runs the tests, not you.** `coco_goal_verify_start({ goalId, expectedSha: })` → then poll `coco_goal_verify_result({ goalId, runId })` until `status:"done"` (coco recorded `verdict` pass|fail from the exit code and advanced — read `nextAction`) or `status:"aborted"` (HEAD moved / tree dirtied during the run — re-`coco_goal_status` and start over). You do NOT report a verdict. Needs a committed `coco.config.json` with `verify.testCommand`; if it's missing, `verify_start` errors — tell the user to add it (no agent fallback). - - **`wrong-branch`** → you are not on the goal branch (e.g. still on `main`). **Do NOT edit or test here.** `git checkout coco/` first, then re-status. + - **`wrong-branch`** → you are not on the goal branch. **Do NOT edit or test here.** `git checkout coco/` first, then re-status. - **`commit-or-revert`** → commit the work or discard it, then re-status. - - **`rebase-needed`** → rebase the branch onto `main`, then re-status (expect fresh review/verify — content changed). + - **`rebase-needed`** → rebase the branch onto the goal's `base`, then re-status (expect fresh review/verify — content changed). - **`escalate-human`** / **`stuck`** (via `coco_health`) → STOP, summarize the blocker for the user. - **`merge-gate`** → STOP the loop. First read `status.autoMergeAllowed`: **If `autoMergeAllowed` (the goal opted in via `--auto`):** try the auto-merge — `coco_merge({ repoDir, goalId, expectedSha: })`: - `{ merged:true, mergedSha }` → **announce the merged SHA**, then go to step 6. - `{ merged:false, next:"continue-loop", reason }` → a transient gate (rebase / re-review / re-verify): re-`coco_goal_status` and resolve — do NOT hand to a human. - - `{ merged:false, next:"human-merge", reason }` → consent/risk sent it back to a human (e.g. sensitive paths, oversized diff, no tests). Fall through to the human path and surface `reason`. + - `{ merged:false, next:"human-merge", reason, humanCommand? }` → consent/risk/verify-policy sent it back to a human. Fall through to the human path and surface `reason`. **Human path (default, or after a `human-merge` fallback):** present the **merge card** — `goalId`, `repoDir`, target branch (``), HEAD SHA, `review: clean`, `verify: pass` (plus any auto-merge `reason`) — then **propose the exact command as an executable step** (this is the consent checkpoint): `coco merge --goal ` *(runs in ``)* + If `reason` mentions `verify.testCommand` / `ack-verify-policy-change`, use: + + `coco merge --goal --ack-verify-policy-change` *(runs in ``)* + - The human approving that command **is** the merge consent — you run the CLI *only* under that approval. Keep `--goal ` in the command so approval is per-goal, never blanket. - `coco merge` prints JSON and **always exits 0 — read the payload, not the exit code**: `{"merged":true}` → step 6; `{"merged":false,"reason":"…"}` → surface + STOP, re-`coco_goal_status` and resolve (don't retry blindly); non-zero / thrown (e.g. `no active goal`) → STOP and surface. - **Auto-approve caveat:** in a no-prompt / bypass harness this runs with no human pause — i.e. it becomes auto-merge *without* the `--auto` opt-in. If you know the session auto-approves, tell the human before proposing. @@ -66,5 +70,5 @@ Call `coco_health({ repoDir, goalId })` any time to get a verdict (`healthy / st - coco tools reach the referee via `[mcp_servers.coco]` in `~/.codex/config.toml`; Oracle via `[mcp_servers.oracle]` (`consult`). Tool names may appear server-prefixed (e.g. `coco__coco_goal_status`). - One goal per repo at a time. `maxFixRounds` (default 5) bounds fix attempts before `escalate-human`. -- **Context delivery to Oracle.** Attach `git diff main...HEAD`, not whole files. For a small review (sources/diff under ~150 lines) **paste them inline in the prompt** rather than uploading — browser attachment upload is slower and occasionally times out; inline is reliable and cheaper. Reserve `browserAttachments:"always"` for genuinely large diffs. +- **Context delivery to Oracle.** Attach `git diff ...HEAD`, not whole files. For a small review (sources/diff under ~150 lines) **paste them inline in the prompt** rather than uploading — browser attachment upload is slower and occasionally times out; inline is reliable and cheaper. Reserve `browserAttachments:"always"` for genuinely large diffs. - **Stuck-detection.** coco escalates to `stuck` two ways: `maxFixRounds` blocking reviews, OR the **same failure repeating 3×** (it hashes your `evidence`). So on a `blocking` review or a `fail` verify, put the **concrete failure detail** in `evidence` — **one or two sentences**: the specific blocking reason or the failing test line, NOT a narrative recap. That's what coco fingerprints, and it keeps `.coco/goals/*.json` small (verbose evidence has bloated goal state past 200 KB). Distinct failures each round reset the counter; a `clean`/`pass` clears it. From f3e7faca324e80e0a59e7502bb33e9e038acbc5e Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:13:32 +0930 Subject: [PATCH 30/66] docs: refresh daily workflow for setup eval and policy ack --- docs/codex-daily-workflow.md | 44 ++++++++++++++++++++++++++++++++---- 1 file changed, 39 insertions(+), 5 deletions(-) diff --git a/docs/codex-daily-workflow.md b/docs/codex-daily-workflow.md index 29a303f..0662ee4 100644 --- a/docs/codex-daily-workflow.md +++ b/docs/codex-daily-workflow.md @@ -11,11 +11,25 @@ This guide is the practical path for using coco every day from Codex.app. pnpm build ``` -2. Configure the coco MCP server in Codex. Start from `.codex/config.toml.example` and copy the relevant blocks into `~/.codex/config.toml`. +2. Dry-run the Codex setup helper: -3. Install the skills in your agent skills directory. Keep the installed skills in sync with this repository when developing coco itself. + ```sh + coco setup codex + ``` + + It reports the `~/.codex/config.toml` MCP change and the skill directories it would sync into `~/.agents/skills`. + +3. Apply the setup when the dry-run looks right: + + ```sh + coco setup codex --apply + ``` + + You can override paths with `--config ` and `--skills-dir `. -4. Run: +4. Configure Oracle MCP. `.codex/config.toml.example` intentionally leaves Oracle as a local placeholder because different machines install Oracle differently. + +5. Run: ```sh coco doctor @@ -64,7 +78,15 @@ Use `--auto` only when you intentionally grant forward consent for this one goal $coco-loop --auto ``` -Auto-merge is still gated by clean review, coco-owned verify, base ancestry, risk policy, and per-goal consent. Risk fallback returns to the human merge path. +Auto-merge is still gated by clean review, coco-owned verify, base ancestry, risk policy, unchanged verification policy, and per-goal consent. Risk or verify-policy fallback returns to the human merge path. + +## Base branch + +New goals branch from `workflow.baseBranch` in `coco.config.json` when set. Otherwise coco resolves the repo default branch (`origin/HEAD`, then `main`/`master`/`trunk`/`develop`, then the current branch). Use an explicit base only for unusual work: + +```sh +coco goal start --objective "..." --base release/1.2 +``` ## Human merge checkpoint @@ -74,6 +96,12 @@ The normal loop ends at `merge-gate`. The agent should present the exact command coco merge --goal ``` +If the goal changed `verify.testCommand`, coco refuses the default merge and requires the human to approve the explicit policy acknowledgement: + +```sh +coco merge --goal --ack-verify-policy-change +``` + Approving that exact command is the merge consent. Do not approve blanket or always-allow merge behavior. ## Self-improvement @@ -94,12 +122,18 @@ Before a branch is ready for review, run: pnpm ci ``` -For referee-critical changes, also run targeted tests for the exact invariant being changed or protected. +`pnpm ci` runs typecheck, tests, the deterministic `coco eval` safety fixtures, and build. For referee-critical changes, also run targeted tests for the exact invariant being changed or protected. + +## Privacy and platform notes + +- Read `docs/privacy-model.md` before sending roadmap/background material to Oracle. +- Read `docs/platform-support.md` before relying on non-macOS/Linux behavior. ## Troubleshooting checklist - `coco doctor` reports `oracle MCP` missing: configure Oracle before plan/review gates. - `goal status` warns about `verify.testCommand`: set `verify.testCommand` in committed `coco.config.json`. +- `merge` refuses with `ack-verify-policy-change`: verify policy changed in the goal diff; inspect it, then approve the explicit acknowledgement command only if intended. - `health` reports `review-unavailable`: fix Oracle login/wiring, then resume with `coco_goal_op_clear`. - `health` reports `stalled`: inspect the current `nextAction`; do not guess the phase from chat memory. - Verify is stuck running: use `coco doctor clean` for old terminal/orphaned verify runs; do not delete live goal state. From d2846d96fa8b43932efd2490d479e8b8943832f1 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:14:12 +0930 Subject: [PATCH 31/66] docs: add privacy model --- docs/privacy-model.md | 59 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 docs/privacy-model.md diff --git a/docs/privacy-model.md b/docs/privacy-model.md new file mode 100644 index 0000000..3159990 --- /dev/null +++ b/docs/privacy-model.md @@ -0,0 +1,59 @@ +# Privacy model + +coco is local-first, but the full loop can send selected context to Oracle for plan/review. Treat every boundary explicitly. + +## Local-only data + +These artifacts are local runtime/project data and should not be pasted into Oracle or external services as raw dumps: + +- `.coco/**`, including goal ledgers, verify-run cache, incidents, and audit logs. +- `.coco/audit.ndjson` and `.coco/incidents.ndjson`. +- `.coco-store/**` cards with `visibility: "local"`. +- Improve specs and audit-derived self-improvement notes. +- Secrets, credentials, private keys, env files, tokens, browser profiles, and local config with account data. + +## Shared data + +These may be sent to Oracle only when the user/skill intentionally includes them: + +- The current task objective. +- The bounded `coco-store pack` brief. +- `.coco-store` cards marked `visibility: "shared"`. +- A bounded background excerpt supplied through `coco-store pack --background` or `--background-stdin`. +- A committed diff such as `git diff ...HEAD` for review. + +## Built-in guards + +`coco-store pack` protects the most common accidents: + +- Background files must resolve inside the repo. +- Symlink escapes are refused. +- Secret-looking path segments are refused. +- Binary files are refused. +- Background is head-bounded by line and byte caps. +- Local cards are excluded from Oracle briefs. + +These guards are necessary but not sufficient. Inline text from the current conversation and roadmap content can still contain sensitive data. The agent and human should keep those concise and scrubbed. + +## Oracle prompt rules + +When asking Oracle to plan/review: + +1. Prefer the diff and small, relevant excerpts over whole files. +2. Do not paste `.coco/**` raw state. +3. Do not paste local improve specs or audit details into external research prompts. +4. Do not include secrets even when they are in tracked files. +5. Mark uncertain privacy decisions as blockers for the human rather than guessing. + +## Self-improvement research + +`$coco-improve` may use external web research only through the static, code-defined `researchTopic` attached to a fired safe signal. It must not include repo names, paths, goal IDs, audit details, timestamps, or local file names in the search query. + +## Review checklist + +Before merging changes that affect privacy boundaries, confirm: + +- New pack/context paths preserve the `visibility: "local"` boundary. +- Any new external call has a documented input surface. +- Secret-looking paths still fail closed. +- Tests cover path traversal, symlink escape, local-card exclusion, and bounded background behavior. From 000948bee66eec131b9912489ddc8eabe28e87fb Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:14:29 +0930 Subject: [PATCH 32/66] docs: document platform support --- docs/platform-support.md | 45 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 docs/platform-support.md diff --git a/docs/platform-support.md b/docs/platform-support.md new file mode 100644 index 0000000..fd0235c --- /dev/null +++ b/docs/platform-support.md @@ -0,0 +1,45 @@ +# Platform support + +coco is currently a **macOS/Linux-first local CLI + MCP tool**. + +## Supported path + +The supported daily path is: + +- macOS, especially Codex.app. +- Linux CI and Linux developer shells. +- Node.js 20 or newer. +- Git available on `PATH`. +- POSIX shell available as `/bin/sh` for coco-owned verify runs. +- `ps` available for local lock-holder identity checks. + +GitHub Actions runs the verification gate on Ubuntu and macOS. + +## Best-effort / not yet guaranteed + +Windows native shells are not yet a guaranteed target. Several implementation details are intentionally POSIX-oriented today: + +- `verifyStart` runs the configured command through `/bin/sh` and uses process-group termination for timeouts. +- `lock.ts` uses `ps` to distinguish a live lock holder from a stale PID. +- Hook installation paths are tailored to Codex/Claude local config layouts. +- Path/secret checks normalize common separators but have not been validated as a full Windows support contract. + +Windows users should run coco through WSL until a Windows adapter is implemented and tested. + +## Portability roadmap + +To support Windows natively, add adapters before changing behavior in place: + +1. `ProcessRunner` for shell/process-group differences. +2. `LockIdentity` for platform-specific PID/start-time checks. +3. `PathPolicy` tests for Windows drive letters, UNC paths, and case folding. +4. CI on `windows-latest`. +5. A documented verify timeout strategy that can terminate process trees reliably. + +## Review checklist + +For platform-sensitive changes, confirm: + +- No new hard dependency on a shell/tool without doctor coverage. +- New paths use repo-relative or realpath-normalized checks at trust boundaries. +- CI covers the intended platform before the README claims support. From 91a7004c544bf066999e7b092098ee1d547b42eb Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:15:58 +0930 Subject: [PATCH 33/66] docs: describe completed safety and productization improvements --- README.md | 38 +++++++++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index d329e85..3332b11 100644 --- a/README.md +++ b/README.md @@ -24,21 +24,26 @@ As of **v0.4**, coco also **improves itself**: `coco-audit` records every meanin ## What's new -- **Daily Codex hardening** — root `AGENTS.md` now captures durable repo/agent guidance, `.codex/config.toml.example` shows the local MCP wiring, and `docs/codex-daily-workflow.md` gives the Mac Codex.app daily path. -- **`coco-mcp` package bin** — the npm package now exposes `coco-mcp` alongside `coco` and `coco-store`, matching the documented MCP setup path. -- **CI-ready verification** — `pnpm typecheck`, `pnpm ci`, and a GitHub Actions matrix run typecheck + tests + build on Linux and macOS. -- **`coco-improve` — self-improving loop (propose-only)** — `$coco-improve` reflects over the `coco audit` corpus (with an **insufficient-data gate** so a thin history can't manufacture findings), forms ONE incremental, evidence-backed hypothesis, and archives it as a **local** improve-spec + one loop-sized backlog task for a human to run through `$coco-loop`. It never builds, merges, or edits live skills. It can ground a proposal in **bounded, cited web research** — a *code-controlled* query, so nothing about coco leaks into a search — treating any external best-practice as *evidence to test, never proof*. A **code-owned guard** (`coco improve check`) refuses any change targeting the referee / metrics / store — enforced at BOTH propose time and, via an `improveOrigin` flag **frozen at goal start**, at **merge** time (even if a task under-declares what it touches). A self-improvement can never quietly weaken coco's own gate. -- **`coco-audit` — automatic trajectory capture** — every meaningful loop/goal action (reviews, fixes, verify results, Oracle outages, merges) is recorded to a local, gitignored `.coco/audit.ndjson` at the domain chokepoint. Deterministic, **best-effort** (a logging failure never breaks the referee), and **redacted** (structural facts only — no evidence text). `coco audit report` aggregates it: fix-rounds (by distinct blocking tree, matching the epoch model), verify failures, Oracle outages, and verify→merge latency — the signal for evolving the loop. -- **`coco doctor` — one-shot health check** — a read-only, **no-LLM** diagnostic that aggregates environment (node/git/version), repo setup (init, `verify.testCommand`), wiring (merge-guard hooks, coco + Oracle MCP, watchdog), active-goal health, and data hygiene. `coco doctor clean` reclaims stale verify-run cache — **dry-run by default**, `--apply` to delete, and only terminal/orphaned runs (never a live goal's runs, the goal ledger, or the audit log). -- **Native `◈ coco` progress cards** — every layer surfaces a consistent, fenced **checkpoint card** so you can watch progress natively in the Codex app (assistant markdown is the only progress surface it renders — there's no live goal HUD there yet). `$coco-loop` echoes a loop checkpoint (*checkpoint · verified · remaining · next*, stamped with `goalId · sha · nextAction`) from `coco_goal_status`'s additive, versioned `progress` field — on each state transition, not every poll. `coco-store status` renders a **project-pulse** card (*backlog by status · spec completion · roadmap*); `$coco-goal` shows its self-reported pipeline phase. One shared visual language across all three; the human merge command stays **outside** the card as a clear, separate approval step. -- **`coco init` scaffolds a starter `coco.config.json`** and goal status **warns early** when `verify.testCommand` is unset — so a fresh repo learns about the coco-owned verify gate up front, instead of stalling at it after plan/implement/review work is already sunk. init force-tracks its own config even past repo ignore rules, and never overwrites a config you already have. +- **Daily Codex hardening** — root `AGENTS.md` captures durable repo/agent guidance, `.codex/config.toml.example` shows local MCP wiring, `coco setup codex` dry-runs/applies MCP + skill setup, and `docs/codex-daily-workflow.md` gives the Mac Codex.app daily path. +- **Referee state hardening** — persisted `.coco/goals/*.json` is schema-validated and stamped with a goal schema version instead of being blindly cast from JSON. +- **Configurable base branch + collision-safe goal ids** — new goals use `workflow.baseBranch` / repo default branch, with explicit `--base` override; repeated same-minute objectives get a safe numeric suffix instead of colliding with existing goal files/branches. +- **Verification-policy acknowledgement** — a goal that changes `verify.testCommand` can still be reviewed/verified, but human merge now requires explicit `--ack-verify-policy-change`; auto-merge falls back to that human path. +- **Additive auto-merge policy globs** — user `autoMerge.sensitiveGlobs` / `testGlobs` add to conservative defaults unless `replaceDefault*Globs` is explicitly set. +- **Command registry + deterministic evals** — `coco commands` exposes command metadata and `coco eval` runs lightweight deterministic safety-regression fixtures; `pnpm ci` includes the eval gate. +- **`coco-mcp` package bin** — the npm package exposes `coco-mcp` alongside `coco` and `coco-store`, matching the documented MCP setup path; the MCP server reports the package version. +- **CI-ready verification** — `pnpm typecheck`, `pnpm test`, `pnpm eval`, `pnpm build`, and `pnpm ci` are wired into a GitHub Actions matrix on Linux and macOS. +- **Privacy and platform docs** — `docs/privacy-model.md` documents what may leave the machine; `docs/platform-support.md` documents the current macOS/Linux support contract. +- **`coco-improve` — self-improving loop (propose-only)** — `$coco-improve` reflects over the `coco audit` corpus (with an **insufficient-data gate** so a thin history can't manufacture findings), forms ONE incremental, evidence-backed hypothesis, and archives it as a **local** improve-spec + one loop-sized backlog task for a human to run through `$coco-loop`. It never builds, merges, or edits live skills. +- **`coco-audit` — automatic trajectory capture** — every meaningful loop/goal action (reviews, fixes, verify results, Oracle outages, merges) is recorded to a local, gitignored `.coco/audit.ndjson` at the domain chokepoint. +- **`coco doctor` — one-shot health check** — a read-only, **no-LLM** diagnostic that aggregates environment (node/git/version), repo setup (init, `verify.testCommand`), wiring (merge-guard hooks, coco + Oracle MCP, watchdog), active-goal health, and data hygiene. `coco doctor clean` reclaims stale verify-run cache — **dry-run by default**, `--apply` to delete, and only terminal/orphaned runs. +- **Native `◈ coco` progress cards** — every layer surfaces a consistent, fenced **checkpoint card** so you can watch progress natively in the Codex app. - **`coco-store` PM surface** — `list --group-by`, `progress` (backlog by status, grouped by spec), `link` (with a content-addressed links-merge), and `viz` (a structural mermaid project graph). -- **`$coco-goal` (CEO layer)** — turns a weak intent into an archivable GoalSpec and promotes loop-sized steps to the backlog, then stops for you. - **Context `pack`** wires the store → loop: the loop reads a bounded coco-store brief before it plans. ## Prerequisites - **Node.js ≥ 20.** +- **macOS or Linux.** Windows native support is not guaranteed yet; use WSL. See `docs/platform-support.md`. - **An MCP-aware coding agent** — [OpenAI Codex](https://developers.openai.com/codex) or [Claude Code](https://claude.ai/code) — with the `coco` (and `oracle`) MCP servers registered and the `coco-goal` / `coco-store` / `coco-loop` skills installed under the agent's skills dir. - **A ChatGPT Pro subscription + Oracle — required for the review brain.** coco's plan/review gate runs **GPT‑5.x‑Pro** through the [`@steipete/oracle`](https://github.com/steipete/oracle) lane, which drives a logged-in **ChatGPT Pro** browser session (the `consult` MCP tool). Without an active **ChatGPT Pro** subscription and Oracle configured, the CLIs still run, but the Oracle-gated plan/review steps can't — by design coco then **fails to the human, never a false green**. @@ -62,9 +67,11 @@ cat .codex/config.toml.example cat docs/codex-daily-workflow.md ``` -Then copy the relevant MCP blocks into `~/.codex/config.toml`, install/sync the skills into your agent skills directory, and run: +Then dry-run/apply setup and check health: ```sh +coco setup codex +coco setup codex --apply coco doctor ``` @@ -80,6 +87,9 @@ The daily path is: ```sh cd your-repo coco init # bootstrap .coco/ + a starter coco.config.json on a clean main +coco setup codex # dry-run local Codex MCP + skill setup +coco commands # command registry / effects overview +coco eval # deterministic safety-regression fixtures # ... the loop is normally driven by the $coco-loop skill, not by hand ... coco-store progress # backlog by status, grouped by spec coco-store status # native ◈ project-pulse card (backlog · specs · roadmap) @@ -96,11 +106,17 @@ coco improve check # refuse edits to the referee/metrics/sto pnpm install pnpm typecheck pnpm test +pnpm eval pnpm build pnpm ci ``` -`pnpm ci` is the pre-PR gate: typecheck, test, then build. GitHub Actions runs the same gate on Linux and macOS. +`pnpm ci` is the pre-PR gate: typecheck, test, deterministic evals, then build. GitHub Actions runs the same gate on Linux and macOS. + +## Privacy and platform + +- Read `docs/privacy-model.md` before adding new context/Oracle surfaces. +- Read `docs/platform-support.md` before claiming support beyond macOS/Linux. ## Credits From d24e13b2e1b065031905686716d7be5fa998d41d Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:16:47 +0930 Subject: [PATCH 34/66] docs: update agent guidance for new gates --- AGENTS.md | 39 +++++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 12 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 2fd6cd1..86e50c5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,10 +5,11 @@ This repository builds `coco`, a small loop-engineering referee for AI coding ag ## Project shape - `src/commands/**` contains CLI/domain command entry points. -- `src/gate.ts`, `src/epoch.ts`, `src/state.ts`, `src/git.ts`, `src/lock.ts`, `src/oracleVerdict.ts`, `src/commands/goal*.ts`, `src/commands/verify.ts`, and `src/commands/merge.ts` are referee-critical. +- `src/gate.ts`, `src/epoch.ts`, `src/state.ts`, `src/goalSchema.ts`, `src/git.ts`, `src/lock.ts`, `src/oracleVerdict.ts`, `src/commands/goal*.ts`, `src/commands/verify.ts`, and `src/commands/merge.ts` are referee-critical. - `src/mcp/**` exposes the same referee through MCP tools. - `src/store/**` is the PM/knowledge layer. It must not import or mutate goal/referee state. - `src/improve/**` implements propose-only self-improvement guards and audit-derived signals. +- `src/commands/eval.ts` contains deterministic safety-regression fixtures. Add a fixture when fixing a state-machine or false-green class. - `skills/**` contains the user-facing Codex/Claude skills. - `.coco/**` is local runtime state and must not be committed. - `.coco-store/**` is the local PM store. Treat audit-derived/self-improvement content as local unless code explicitly marks it shared. @@ -21,24 +22,36 @@ Use pnpm. pnpm install pnpm typecheck pnpm test +pnpm eval pnpm build pnpm ci ``` `pnpm ci` is the expected pre-PR verification command. If a change touches a safety-critical gate, run the most specific affected tests as well as `pnpm ci`. +Useful local product commands: + +```sh +coco commands +coco setup codex # dry-run Codex MCP + skill setup +coco setup codex --apply # apply local setup +coco eval # deterministic safety fixtures +``` + ## Safety invariants -Do not weaken these without an explicit, human-authored referee-change goal and tests that prove the new behavior: +Do not weaken these without an explicit, human-authored referee-change goal and tests/evals that prove the new behavior: 1. A review verdict must come from strict Oracle output parsing; never add a caller-asserted clean verdict path. 2. Verify is coco-owned. The agent must not self-report `pass`. 3. Review and verify are bound to the current HEAD/tree and require a clean working tree. 4. A tree that was blocking/failing must not be silently re-blessed without new code and fresh review/verify. -5. Merge requires active goal, correct branch, clean tree, implement in the current epoch, clean review, passing verify, and base ancestry. -6. Auto-merge is opt-in per goal and must never loosen the human merge path. -7. Improve-origin changes must not touch protected referee, metrics, store, or improve-self paths. -8. Store/pack must preserve the privacy boundary: local cards and secret-looking files must not be sent to Oracle. +5. Persisted goal state is untrusted input; load it through `goalSchema`, not raw casts. +6. Merge requires active goal, correct branch, clean tree, implement in the current epoch, clean review, passing verify, and base ancestry. +7. A goal that changes `verify.testCommand` requires explicit human acknowledgement with `--ack-verify-policy-change`; auto-merge must fall back to the human path. +8. Auto-merge is opt-in per goal and must never loosen the human merge path. +9. Improve-origin changes must not touch protected referee, metrics, store, or improve-self paths. +10. Store/pack must preserve the privacy boundary: local cards and secret-looking files must not be sent to Oracle. ## Coding style @@ -46,20 +59,22 @@ Do not weaken these without an explicit, human-authored referee-change goal and - Keep LLM/Oracle assumptions in skills or explicit integration seams, not in the referee core. - Fail closed at trust boundaries: malformed state, ambiguous verdicts, missing verify config, path traversal, stale/forged verify runs, and protected-path changes should stop the loop rather than infer success. - Avoid broad rewrites of `skills/**`; make small, reviewable deltas because skill text is part of the safety surface. -- Add regression tests for every bug fix. For referee changes, include the concrete false-green or unsafe-transition scenario. +- Add regression tests for every bug fix. For referee changes, include the concrete false-green or unsafe-transition scenario and consider adding a `coco eval` fixture. ## Branch and merge expectations - Work on a branch; do not commit directly to `main`. - Do not merge from an agent session unless the user explicitly approves the exact merge action for that goal/branch. +- If coco returns a `humanCommand`, present that exact command. If verify policy changed, the command must include `--ack-verify-policy-change`. - A ready PR should explain the safety impact, affected commands/tools, and verification run. ## Daily Codex workflow For normal development in Codex.app: -1. Run `coco doctor` before a long session. -2. Use `$coco-goal` for vague or multi-step work. -3. Use `$coco-loop` for one loop-sized implementation task. -4. Use `$coco-store status` or `coco-store progress` for project state. -5. Use `$coco-improve` only to propose one evidence-backed improvement; it must not build, edit, or merge the proposal itself. +1. Run `coco setup codex` once as a dry-run, then `coco setup codex --apply` if the paths are correct. +2. Run `coco doctor` before a long session. +3. Use `$coco-goal` for vague or multi-step work. +4. Use `$coco-loop` for one loop-sized implementation task. +5. Use `$coco-store status` or `coco-store progress` for project state. +6. Use `$coco-improve` only to propose one evidence-backed improvement; it must not build, edit, or merge the proposal itself. From fa505acd720737e7e4b30e6628107c0fda8cbf65 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:19:48 +0930 Subject: [PATCH 35/66] ci: use supported Node matrix versions --- .github/workflows/ci.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 730bb3e..0fdaf56 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -16,7 +16,10 @@ jobs: fail-fast: false matrix: os: [ubuntu-latest, macos-latest] - node: [20, 22] + # Node 20 remains the package minimum, but GitHub-hosted Actions runners now reject + # Node 20 setup without an insecure compatibility flag. Keep CI on supported current + # runtimes while package.json documents the install floor. + node: [22, 24] steps: - name: Checkout From d0882b82db3cd73a40ea7709f1035e6927e756e2 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:42:28 +0930 Subject: [PATCH 36/66] feat: validate audit records and add feedback events --- src/audit.ts | 139 ++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 126 insertions(+), 13 deletions(-) diff --git a/src/audit.ts b/src/audit.ts index 296aa8d..41e3d21 100644 --- a/src/audit.ts +++ b/src/audit.ts @@ -1,5 +1,7 @@ +import { createHash } from 'node:crypto'; import { appendFileSync, existsSync, mkdirSync, readFileSync } from 'node:fs'; import { join } from 'node:path'; +import { z, ZodError } from 'zod'; import { cocoDir } from './paths.js'; import type { GoalState } from './types.js'; @@ -12,11 +14,21 @@ import type { GoalState } from './types.js'; export const AUDIT_SCHEMA_VERSION = 1; +export const AUDIT_FEEDBACK_KINDS = [ + 'goal-quality', + 'implementation-quality', + 'loop-friction', + 'review-quality', + 'verification-quality', + 'status-clarity', +] as const; +export type AuditFeedbackKind = (typeof AUDIT_FEEDBACK_KINDS)[number]; + export interface AuditRecord { v: number; // schema version (AUDIT_SCHEMA_VERSION) — lets readers evolve the shape safely at: string; // ISO timestamp of the action goalId: string; - action: string; // the operation label, e.g. 'goal-start' | 'record:review:blocking' | 'verify-start:' | 'merge' | 'auto-merge' | 'oracle-unavailable:review:oracle-timeout' + action: string; // e.g. 'goal-start' | 'record:review:blocking' | 'merge' | 'feedback:goal-quality' state: string; // goal lifecycle at write time (active | achieved | …) — captures the merge→achieved transition events: number; // event-ledger length — a cheap monotonic progress signal phase?: string; // record:* only — the phase of the just-appended event @@ -24,6 +36,61 @@ export interface AuditRecord { commit?: string; // record:* only — HEAD sha of the event tree?: string; // record:* only — git tree hash of the event failCount?: number; // fingerprint stuck-detection counter, when present + kind?: AuditFeedbackKind; // feedback:* only — structured human feedback kind + rating?: number; // feedback:* only — 1..5, higher is better + tags?: string[]; // feedback:* only — normalized short tags, no prose + noteHash?: string; // feedback:* only — hash of local note, never the note itself + noteLength?: number; // feedback:* only — lets us distinguish empty vs substantive feedback without storing prose +} + +const auditFeedbackKindSchema = z.enum(AUDIT_FEEDBACK_KINDS); +const auditRecordSchema = z + .object({ + v: z.number().int().positive(), + at: z.string().min(1), + goalId: z.string().min(1), + action: z.string().min(1), + state: z.string().min(1), + events: z.number().int().nonnegative(), + phase: z.string().optional(), + verdict: z.string().optional(), + commit: z.string().optional(), + tree: z.string().optional(), + failCount: z.number().int().nonnegative().optional(), + kind: auditFeedbackKindSchema.optional(), + rating: z.number().int().min(1).max(5).optional(), + tags: z.array(z.string()).optional(), + noteHash: z.string().optional(), + noteLength: z.number().int().nonnegative().optional(), + }) + .passthrough(); + +export interface AuditInvalidLine { + line: number; + reason: string; + rawHash: string; +} +export interface ReadAuditDetailed { + records: AuditRecord[]; + invalid: AuditInvalidLine[]; + totalLines: number; +} + +function hashText(raw: string): string { + return createHash('sha256').update(raw).digest('hex').slice(0, 16); +} + +function explainZod(e: ZodError): string { + return e.issues.map((i) => `${i.path.join('.') || ''}: ${i.message}`).join('; '); +} + +export function parseAuditRecord(raw: unknown): AuditRecord { + try { + return auditRecordSchema.parse(raw) as AuditRecord; + } catch (e) { + if (e instanceof ZodError) throw new Error(`audit schema invalid (${explainZod(e)})`); + throw e; + } } export function auditPath(repo: string): string { @@ -54,13 +121,51 @@ export function buildAuditRecord(goal: GoalState, operation: string): AuditRecor return rec; } +function normalizeTags(tags?: string[]): string[] | undefined { + const out = [...new Set((tags ?? []).map((t) => t.trim().toLowerCase()).filter(Boolean).map((t) => t.slice(0, 40)))]; + return out.length ? out : undefined; +} + +export function buildFeedbackRecord(input: { + goalId: string; + kind: AuditFeedbackKind; + rating: number; + tags?: string[]; + note?: string; + at?: Date; +}): AuditRecord { + if (!input.goalId.trim()) throw new Error('coco audit feedback: --goal is required'); + if (!AUDIT_FEEDBACK_KINDS.includes(input.kind)) throw new Error(`coco audit feedback: --kind must be ${AUDIT_FEEDBACK_KINDS.join('|')}`); + if (!Number.isInteger(input.rating) || input.rating < 1 || input.rating > 5) throw new Error('coco audit feedback: --rating must be an integer 1..5'); + const note = input.note?.trim(); + return parseAuditRecord({ + v: AUDIT_SCHEMA_VERSION, + at: (input.at ?? new Date()).toISOString(), + goalId: input.goalId.trim(), + action: `feedback:${input.kind}`, + state: 'feedback', + events: 0, + kind: input.kind, + rating: input.rating, + ...(normalizeTags(input.tags) ? { tags: normalizeTags(input.tags) } : {}), + ...(note ? { noteHash: hashText(note), noteLength: note.length } : {}), + }); +} + +export function appendAuditFeedback(repo: string, input: Parameters[0]): AuditRecord { + const rec = buildFeedbackRecord(input); + appendAudit(repo, rec); + return rec; +} + /** Append one audit line. Best-effort: NEVER throws — auditing must not break the referee. */ export function appendAudit(repo: string, record: AuditRecord): void { try { + const normalized = parseAuditRecord(record); // cheap fail-closed guard: only valid audit shapes are written mkdirSync(cocoDir(repo), { recursive: true }); - appendFileSync(auditPath(repo), `${JSON.stringify(record)}\n`); + appendFileSync(auditPath(repo), `${JSON.stringify(normalized)}\n`); } catch { - // observability is best-effort — a failed write is silently dropped, the loop is unaffected + // observability is best-effort — a failed/invalid write is silently dropped, the loop is unaffected } } @@ -74,18 +179,26 @@ export function auditGoalWrite(repo: string, goal: GoalState, operation: string) } /** Read the audit stream, tolerant of malformed/partial lines (append-only files can tear on crash). */ -export function readAudit(repo: string): AuditRecord[] { +export function readAuditDetailed(repo: string): ReadAuditDetailed { const p = auditPath(repo); - if (!existsSync(p)) return []; - const out: AuditRecord[] = []; - for (const line of readFileSync(p, 'utf8').split('\n')) { + if (!existsSync(p)) return { records: [], invalid: [], totalLines: 0 }; + const records: AuditRecord[] = []; + const invalid: AuditInvalidLine[] = []; + const lines = readFileSync(p, 'utf8').split('\n'); + let totalLines = 0; + lines.forEach((line, i) => { const t = line.trim(); - if (!t) continue; + if (!t) return; + totalLines++; try { - out.push(JSON.parse(t) as AuditRecord); - } catch { - // skip a torn/malformed line rather than failing the whole read + records.push(parseAuditRecord(JSON.parse(t) as unknown)); + } catch (e) { + invalid.push({ line: i + 1, reason: (e as Error).message, rawHash: hashText(t) }); } - } - return out; + }); + return { records, invalid, totalLines }; +} + +export function readAudit(repo: string): AuditRecord[] { + return readAuditDetailed(repo).records; } From f35af18e2df149b6dea99e1c12d0a26749ba7480 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:43:57 +0930 Subject: [PATCH 37/66] feat: add audit validity and feedback summary --- src/commands/audit.ts | 141 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 127 insertions(+), 14 deletions(-) diff --git a/src/commands/audit.ts b/src/commands/audit.ts index 8baf483..d9f0d27 100644 --- a/src/commands/audit.ts +++ b/src/commands/audit.ts @@ -1,13 +1,13 @@ -import { readAudit, type AuditRecord } from '../audit.js'; +import { readAuditDetailed, type AuditFeedbackKind, type AuditRecord } from '../audit.js'; // Deterministic analysis over the captured audit stream (no LLM). Surfaces the signals coco-improve -// will later reflect on: where the loop churns (fix rounds), where it fails (verify), where Oracle -// falls over (outages), and how long a green goal waits for the human merge. +// reflects on: where the loop churns (fix rounds), where it fails (verify), where Oracle falls over +// (outages), whether audit itself is trustworthy, and what structured human feedback says. export interface GoalAuditSummary { goalId: string; actions: number; - fixRounds: number; // blocking reviews recorded + fixRounds: number; // distinct blocking review trees verifyFails: number; oracleOutages: number; reachedMerge: boolean; @@ -15,13 +15,39 @@ export interface GoalAuditSummary { verifyToMergeSec?: number; // last clean verify pass → merge/auto-merge latency (human-merge wait) } +export interface FeedbackSummary { + total: number; + negative: number; + avgRating?: number; + byKind: Record; + topTags: [string, number][]; +} + +export interface AuditValidityFailure { + goalId?: string; + line?: number; + code: string; + detail: string; +} + +export interface AuditValidityReport { + ok: boolean; + totalLines: number; + validRecords: number; + invalidRecords: number; + failures: AuditValidityFailure[]; +} + export interface AuditReport { totalRecords: number; totals: { goals: number; fixRounds: number; verifyFails: number; oracleOutages: number; merges: number }; + validity: AuditValidityReport; + feedback: FeedbackSummary; goals: GoalAuditSummary[]; } -const isMerge = (a: string): boolean => a === 'merge' || a === 'auto-merge'; +const isMerge = (a: string): boolean => a === 'merge' || a === 'auto-merge' || a === 'merge:verify-policy-ack'; +const isFeedback = (r: AuditRecord): boolean => r.action.startsWith('feedback:'); /** Fix rounds = distinct trees that got a blocking review — matching the epoch model in src/epoch.ts * (a blocking tree can never later be clean on the SAME tree), NOT the raw count of blocking records @@ -33,21 +59,22 @@ function distinctBlockingTrees(recs: AuditRecord[]): number { } function summarize(goalId: string, recs: AuditRecord[]): GoalAuditSummary { + const nonFeedback = recs.filter((r) => !isFeedback(r)); const s: GoalAuditSummary = { goalId, - actions: recs.length, - fixRounds: distinctBlockingTrees(recs), - verifyFails: recs.filter((r) => r.action === 'record:verify:fail').length, - oracleOutages: recs.filter((r) => r.action.startsWith('oracle-unavailable')).length, - reachedMerge: recs.some((r) => isMerge(r.action) || r.state === 'achieved'), + actions: nonFeedback.length, + fixRounds: distinctBlockingTrees(nonFeedback), + verifyFails: nonFeedback.filter((r) => r.action === 'record:verify:fail').length, + oracleOutages: nonFeedback.filter((r) => r.action.startsWith('oracle-unavailable')).length, + reachedMerge: nonFeedback.some((r) => isMerge(r.action) || r.state === 'achieved'), }; - const last = recs[recs.length - 1]; + const last = nonFeedback[nonFeedback.length - 1]; if (last) s.finalState = last.state; // human-merge latency: last clean verify pass → the FIRST merge at/after it (guards a stray earlier merge record) - const lastPass = [...recs].reverse().find((r) => r.action === 'record:verify:pass'); + const lastPass = [...nonFeedback].reverse().find((r) => r.action === 'record:verify:pass'); if (lastPass) { - const merge = recs.find((r) => isMerge(r.action) && Date.parse(r.at) >= Date.parse(lastPass.at)); + const merge = nonFeedback.find((r) => isMerge(r.action) && Date.parse(r.at) >= Date.parse(lastPass.at)); if (merge) { const dt = (Date.parse(merge.at) - Date.parse(lastPass.at)) / 1000; if (Number.isFinite(dt)) s.verifyToMergeSec = Math.round(dt); @@ -56,16 +83,100 @@ function summarize(goalId: string, recs: AuditRecord[]): GoalAuditSummary { return s; } +function feedbackSummary(recs: AuditRecord[]): FeedbackSummary { + const feedback = recs.filter((r) => isFeedback(r) && typeof r.rating === 'number'); + const byKindRaw = new Map(); + const tags = new Map(); + for (const r of feedback) { + const k = r.kind ?? r.action.replace(/^feedback:/, ''); + const arr = byKindRaw.get(k); + if (arr) arr.push(r); + else byKindRaw.set(k, [r]); + for (const t of r.tags ?? []) tags.set(t, (tags.get(t) ?? 0) + 1); + } + const avg = (xs: AuditRecord[]): number | undefined => { + const nums = xs.map((r) => r.rating).filter((v): v is number => typeof v === 'number'); + return nums.length ? Math.round((nums.reduce((a, b) => a + b, 0) / nums.length) * 10) / 10 : undefined; + }; + const byKind: FeedbackSummary['byKind'] = {}; + for (const [k, xs] of byKindRaw) { + const a = avg(xs); + byKind[k] = { + total: xs.length, + negative: xs.filter((r) => (r.rating ?? 5) <= 2).length, + ...(a !== undefined ? { avgRating: a } : {}), + }; + } + const a = avg(feedback); + return { + total: feedback.length, + negative: feedback.filter((r) => (r.rating ?? 5) <= 2).length, + ...(a !== undefined ? { avgRating: a } : {}), + byKind, + topTags: [...tags.entries()].sort((x, y) => y[1] - x[1] || x[0].localeCompare(y[0])).slice(0, 10), + }; +} + +function auditInvariants(records: AuditRecord[]): AuditValidityFailure[] { + const failures: AuditValidityFailure[] = []; + const byGoal = new Map(); + for (const r of records) { + if (r.action.startsWith('record:review:') && (r.phase !== 'review' || !['clean', 'blocking'].includes(r.verdict ?? ''))) { + failures.push({ goalId: r.goalId, code: 'bad-review-record', detail: `${r.action} must carry phase=review and verdict clean|blocking` }); + } + if (r.action.startsWith('record:verify:') && (r.phase !== 'verify' || !['pass', 'fail'].includes(r.verdict ?? ''))) { + failures.push({ goalId: r.goalId, code: 'bad-verify-record', detail: `${r.action} must carry phase=verify and verdict pass|fail` }); + } + if (r.action.startsWith('feedback:') && (!r.kind || typeof r.rating !== 'number')) { + failures.push({ goalId: r.goalId, code: 'bad-feedback-record', detail: `${r.action} must carry kind and rating` }); + } + const arr = byGoal.get(r.goalId); + if (arr) arr.push(r); + else byGoal.set(r.goalId, [r]); + } + + for (const [goalId, recs] of byGoal) { + let prev = -Infinity; + let sawVerifyPass = false; + for (const r of recs) { + const t = Date.parse(r.at); + if (!Number.isFinite(t)) failures.push({ goalId, code: 'bad-timestamp', detail: `${r.action} has an unparseable timestamp` }); + if (Number.isFinite(t) && t < prev) failures.push({ goalId, code: 'non-monotonic-time', detail: `${r.action} timestamp moved backwards` }); + if (Number.isFinite(t)) prev = t; + if (r.action === 'record:verify:pass') sawVerifyPass = true; + if (isMerge(r.action) && !sawVerifyPass) failures.push({ goalId, code: 'merge-before-verify-pass', detail: `${r.action} appeared before any verify pass` }); + } + } + return failures; +} + +/** Validate audit shape + cross-record invariants. Read-only and deterministic. */ +export function auditValidate(repo: string): AuditValidityReport { + const detailed = readAuditDetailed(repo); + const parseFailures: AuditValidityFailure[] = detailed.invalid.map((i) => ({ line: i.line, code: 'invalid-line', detail: `${i.reason} [${i.rawHash}]` })); + const failures = [...parseFailures, ...auditInvariants(detailed.records)]; + return { + ok: failures.length === 0, + totalLines: detailed.totalLines, + validRecords: detailed.records.length, + invalidRecords: detailed.invalid.length, + failures, + }; +} + /** Aggregate the audit stream per goal (insertion order preserved). Read-only, deterministic. */ export function auditReport(repo: string): AuditReport { - const recs = readAudit(repo); + const detailed = readAuditDetailed(repo); + const recs = detailed.records; const byGoal = new Map(); for (const r of recs) { + if (isFeedback(r)) continue; const arr = byGoal.get(r.goalId); if (arr) arr.push(r); else byGoal.set(r.goalId, [r]); } const goals = [...byGoal.entries()].map(([id, rs]) => summarize(id, rs)); + const validity = auditValidate(repo); return { totalRecords: recs.length, totals: { @@ -75,6 +186,8 @@ export function auditReport(repo: string): AuditReport { oracleOutages: goals.reduce((n, g) => n + g.oracleOutages, 0), merges: goals.filter((g) => g.reachedMerge).length, }, + validity, + feedback: feedbackSummary(recs), goals, }; } From 0d9f58829c531b0d6ec278215f8fa4a311a425d6 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:46:52 +0930 Subject: [PATCH 38/66] feat: expose audit validate and feedback CLI --- src/cli.ts | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/src/cli.ts b/src/cli.ts index 47f78bb..96fd2b5 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -35,8 +35,8 @@ import { goalHealth } from './commands/health.js'; import { guardHook, runGuard } from './commands/guard.js'; import { defaultPaths, installHooks, uninstallHooks } from './commands/installHooks.js'; import { cocoDone, cocoNext } from './commands/backlog.js'; -import { auditReport } from './commands/audit.js'; -import { readAudit } from './audit.js'; +import { auditReport, auditValidate } from './commands/audit.js'; +import { appendAuditFeedback, AUDIT_FEEDBACK_KINDS, readAudit, type AuditFeedbackKind } from './audit.js'; import { cleanDoctor, runDoctor } from './commands/doctor.js'; import { runEval } from './commands/eval.js'; import { listCommandDescriptors } from './commands/registry.js'; @@ -400,6 +400,23 @@ export function main(argv: string[] = process.argv.slice(2)): number { out(values.goal ? recs.filter((r) => r.goalId === values.goal) : recs); return 0; } + if (sub === 'validate') { + const res = auditValidate(repo); + out(res); + return res.ok ? 0 : 1; + } + if (sub === 'feedback') { + const { values } = parseArgs({ + args: rest, + options: { goal: { type: 'string' }, kind: { type: 'string' }, rating: { type: 'string' }, tags: { type: 'string' }, note: { type: 'string' } }, + }); + if (!values.goal || !values.kind || !values.rating) { + throw new Error(`usage: coco audit feedback --goal --kind <${AUDIT_FEEDBACK_KINDS.join('|')}> --rating <1..5> [--tags a,b] [--note "local note"]`); + } + const tags = values.tags ? values.tags.split(',').map((s) => s.trim()).filter(Boolean) : undefined; + out(appendAuditFeedback(repo, { goalId: values.goal, kind: values.kind as AuditFeedbackKind, rating: Number(values.rating), tags, note: values.note })); + return 0; + } if (sub === undefined || sub === 'report') { out(auditReport(repo)); return 0; From a12e939e712f62f2ddf716c8f7fdb45db498fbb4 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:48:52 +0930 Subject: [PATCH 39/66] feat: surface audit validity in doctor --- src/commands/doctor.ts | 31 +++++++------------------------ 1 file changed, 7 insertions(+), 24 deletions(-) diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index 2be79f2..0ca7e13 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -1,7 +1,6 @@ import { existsSync, lstatSync, readdirSync, readFileSync, rmSync, statSync } from 'node:fs'; import { homedir } from 'node:os'; -import { dirname, join } from 'node:path'; -import { fileURLToPath } from 'node:url'; +import { join } from 'node:path'; import { auditPath } from '../audit.js'; import { readVerifyConfig, VERIFY_NOT_CONFIGURED_WARNING } from '../cocoConfig.js'; import { tryGit } from '../git.js'; @@ -10,6 +9,8 @@ import { lockStatus } from '../lock.js'; import { cocoDir, goalsDir, lockPath } from '../paths.js'; import { readGoal } from '../state.js'; import { storeDir } from '../store/paths.js'; +import { cocoVersion } from '../version.js'; +import { auditValidate } from './audit.js'; import { goalHealth } from './health.js'; import { defaultPaths, isGuardInstalled } from './installHooks.js'; import { listWatchdogs } from './watchdog.js'; @@ -61,26 +62,6 @@ function dirSize(dir: string): number { return total; } -/** coco's own version — walk up from this module to the nearest package.json (works in dev + dist). */ -function cocoVersion(): string { - try { - let dir = dirname(fileURLToPath(import.meta.url)); - for (let i = 0; i < 6; i++) { - const p = join(dir, 'package.json'); - if (existsSync(p)) { - const v = (JSON.parse(readFileSync(p, 'utf8')) as { version?: string }).version; - if (v) return v; - } - const parent = dirname(dir); - if (parent === dir) break; - dir = parent; - } - } catch { - // fall through - } - return 'unknown'; -} - /** .gitignore lists `.coco/` (the check goalStart uses to gate `coco init`). Tolerant of a read error. */ function cocoInitialized(repo: string): boolean { try { @@ -109,7 +90,7 @@ function environmentChecks(repo: string): Check[] { const major = Number(process.versions.node.split('.')[0]); const git = tryGit(repo, ['--version']); return [ - { group: 'environment', name: 'node', status: major >= 18 ? 'ok' : 'fail', detail: `${process.version} (need >=18)` }, + { group: 'environment', name: 'node', status: major >= 20 ? 'ok' : 'fail', detail: `${process.version} (need >=20)` }, { group: 'environment', name: 'git', status: git.ok ? 'ok' : 'fail', detail: git.ok ? git.out.trim() : 'git not found on PATH' }, { group: 'environment', name: 'coco version', status: 'ok', detail: cocoVersion() }, ]; @@ -167,8 +148,10 @@ function dataChecks(repo: string, now: number): Check[] { const ls = lockStatus(lockPath(repo), now); const lockDetail = ls.stale ? 'stale (held by a dead process) — `coco doctor clean`' : ls.held ? 'held (op in progress)' : 'free'; const logBytes = fileSize(auditPath(repo)) + fileSize(incidentsPath(repo)); + const audit = auditValidate(repo); return [ { group: 'data', name: 'goals', status: 'ok', detail: goalFiles.length ? JSON.stringify(byState) : 'none' }, + { group: 'data', name: 'audit validity', status: audit.ok ? 'ok' : 'fail', detail: audit.ok ? `${audit.validRecords} valid record(s)` : `${audit.failures.length} failure(s), ${audit.invalidRecords} invalid line(s) — run \`coco audit validate\`` }, { group: 'data', name: 'verify-runs', status: runs.length > 20 ? 'warn' : 'ok', detail: `${runs.length} run(s), ${Math.round(runBytes / 1024)} KB` + (runs.length > 20 ? ' — `coco doctor clean`' : '') }, { group: 'data', name: 'logs', status: 'ok', detail: `audit+incidents ${Math.round(logBytes / 1024)} KB` }, { group: 'data', name: 'lock', status: ls.stale ? 'warn' : 'ok', detail: lockDetail }, @@ -261,5 +244,5 @@ export function cleanDoctor(repo: string, opts: { apply?: boolean } = {}): Clean } } } - return { applied: opts.apply === true, targets, reclaimedBytes: reclaimed }; + return { applied: opts.apply === true, targets, reclaimedBytes: opts.apply ? reclaimed : targets.reduce((n, t) => n + t.bytes, 0) }; } From 6156fad63fcbb9fcb0d0832acf285685d49adafd Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:50:15 +0930 Subject: [PATCH 40/66] feat: feed audit validity and feedback into improve digest --- src/improve/digest.ts | 62 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 56 insertions(+), 6 deletions(-) diff --git a/src/improve/digest.ts b/src/improve/digest.ts index 979e485..51daf0d 100644 --- a/src/improve/digest.ts +++ b/src/improve/digest.ts @@ -1,4 +1,4 @@ -import { auditReport } from '../commands/audit.js'; +import { auditReport, type FeedbackSummary } from '../commands/audit.js'; // coco-improve digest: a DETERMINISTIC (no-LLM, no web) read over the audit corpus that surfaces // the loop's pain signals for the reflective skill to reason about. It reports only STRUCTURAL @@ -6,6 +6,7 @@ import { auditReport } from '../commands/audit.js'; // size so a thin history can't manufacture confident "findings" from noise. const MIN_GOALS = 5; // below this the audit window is too small to claim anything → insufficient-data +const MIN_FEEDBACK = 3; // below this structured feedback is too sparse to claim a product-quality signal // Every proposal that flows from this digest must carry these — the invariant is signal QUALITY, // never throughput. Optimising the wrong metric (fewer fix rounds / faster merge) rewards weaker @@ -15,13 +16,14 @@ export const IMPROVE_ANTI_GOALS: readonly string[] = [ 'Never optimise throughput: time-to-merge, fix-round count, and verify pass-rate reward weaker scrutiny.', 'The referee is off-limits (gate / verdict / verify / merge / risk / metrics). Validate every proposed target with `coco improve check`.', 'Oracle-reliability fixes must NOT loosen retry-once or verdict strictness in the loop skill — improve availability, never scrutiny.', + 'Human feedback is evidence to investigate, not proof; every proposal needs a falsifiable eval or explicit non-regression check.', ]; export interface ImproveSignal { key: string; status: 'signal' | 'clear' | 'insufficient-data'; detail: string; - sample: number; // how many goals/records informed this signal + sample: number; // how many goals/records/feedback entries informed this signal safeToActOn: boolean; // false = diagnostic/observational only; NOT a valid optimisation target researchTopic?: string; // a fired safeToActOn signal only: the GENERIC, code-controlled query to research (never any coco/audit specifics — so nothing private can leak into an external search) } @@ -31,11 +33,14 @@ export interface ImproveSignal { // coco state (names, paths, ids, counts, timestamps). const RESEARCH_TOPICS: Record = { 'oracle-reliability': 'reliable retry, backoff, and resume patterns for flaky LLM or tool calls in agent skill instructions', + 'goal-quality-feedback': 'best practices for writing verifiable software engineering goals for AI coding agents', + 'status-clarity-feedback': 'best practices for progress status and recovery messages in AI coding agent workflows', }; export interface ImproveDigest { - window: { goals: number; records: number }; + window: { goals: number; records: number; feedback: number; invalidAudit: number }; minGoals: number; + minFeedback: number; sufficient: boolean; signals: ImproveSignal[]; antiGoals: readonly string[]; @@ -52,6 +57,27 @@ function signal(key: string, sufficient: boolean, fired: boolean, detail: string return { key, status, detail, sample, safeToActOn, ...(topic ? { researchTopic: topic } : {}) }; } +function fbKind(f: FeedbackSummary, kind: string): { total: number; negative: number; avgRating?: number } { + return f.byKind[kind] ?? { total: 0, negative: 0 }; +} + +function feedbackSignal( + key: string, + f: FeedbackSummary, + kind: string, + label: string, + inherentlySafe: boolean, +): ImproveSignal { + const s = fbKind(f, kind); + const sufficient = s.total >= MIN_FEEDBACK; + const avg = s.avgRating; + const fired = sufficient && (s.negative >= 2 || (avg !== undefined && avg <= 3)); + const detail = sufficient + ? `${label}: ${s.total} feedback entr${s.total === 1 ? 'y' : 'ies'}, avg ${avg ?? 'n/a'}, ${s.negative} negative${fired ? ' — investigate this quality surface' : ''}` + : `${label}: ${s.total}/${MIN_FEEDBACK} feedback entries — insufficient structured feedback`; + return signal(key, sufficient, fired, detail, s.total, inherentlySafe); +} + /** Deterministic pain digest over the audit corpus. Read-only. */ export function improveDigest(repo: string): ImproveDigest { const rep = auditReport(repo); @@ -64,8 +90,8 @@ export function improveDigest(repo: string): ImproveDigest { const avgLatency = latencies.length ? Math.round(latencies.reduce((a, b) => a + b, 0) / latencies.length) : undefined; const signals: ImproveSignal[] = [ - // The one signal safe to act on directly: Oracle reliability. A recurring outage is a loop/skill - // guidance gap (retry/resume), fixing which improves signal availability without touching scrutiny. + // The one original signal safe to act on directly: Oracle reliability. A recurring outage is a + // loop/skill guidance gap (retry/resume), fixing which improves signal availability without touching scrutiny. signal( 'oracle-reliability', sufficient, @@ -74,6 +100,23 @@ export function improveDigest(repo: string): ImproveDigest { goals, true, ), + // Safe product-quality signals: they target skill/workflow clarity, not referee looseness. Every + // promoted task must still pass the protected-path guard and carry an eval/non-regression check. + feedbackSignal('goal-quality-feedback', rep.feedback, 'goal-quality', 'goal quality', true), + feedbackSignal('status-clarity-feedback', rep.feedback, 'status-clarity', 'status clarity', true), + // Diagnostic only: implementation-quality complaints often point at scope/verification/review but + // can tempt throughput optimisation. Use them to investigate; do not optimise the count directly. + feedbackSignal('implementation-quality-feedback', rep.feedback, 'implementation-quality', 'implementation quality', false), + // Audit validity is a trust substrate problem. It is surfaced prominently but not safe for + // coco-improve because audit/metrics files are protected from self-improvement changes. + signal( + 'audit-validity', + true, + !rep.validity.ok, + rep.validity.ok ? 'audit stream valid' : `${rep.validity.failures.length} audit validity failure(s) — self-improvement must pause until a human fixes the audit substrate`, + rep.validity.totalLines, + false, + ), // Diagnostic only: high churn points at a planning/skill gap — investigate the CAUSE; the count // itself must never be optimised (that would reward weaker review). signal( @@ -96,5 +139,12 @@ export function improveDigest(repo: string): ImproveDigest { ), ]; - return { window: { goals, records: rep.totalRecords }, minGoals: MIN_GOALS, sufficient, signals, antiGoals: IMPROVE_ANTI_GOALS }; + return { + window: { goals, records: rep.totalRecords, feedback: rep.feedback.total, invalidAudit: rep.validity.invalidRecords }, + minGoals: MIN_GOALS, + minFeedback: MIN_FEEDBACK, + sufficient, + signals, + antiGoals: IMPROVE_ANTI_GOALS, + }; } From faa5ded77b722d11b15eec8be731cc74ac5b35db Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:51:16 +0930 Subject: [PATCH 41/66] feat: add status edges for rich loop cards --- src/commands/goalStatus.ts | 68 +++++++++++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 4 deletions(-) diff --git a/src/commands/goalStatus.ts b/src/commands/goalStatus.ts index 1e6fb03..15423f2 100644 --- a/src/commands/goalStatus.ts +++ b/src/commands/goalStatus.ts @@ -1,13 +1,30 @@ import { existsSync } from 'node:fs'; import { verifyConfigWarnings } from '../cocoConfig.js'; import { deriveFacts, type DerivedFacts } from '../epoch.js'; -import { gatherLive, headSha } from '../git.js'; +import { currentBranch, gatherLive, headSha } from '../git.js'; import { nextAction, type LiveGit, type NextAction } from '../gate.js'; import { findActiveGoal, goalPath, readGoal } from '../state.js'; +export interface StatusEdge { + kind: + | 'wrong-branch' + | 'dirty-tree' + | 'rebase-needed' + | 'oracle-unavailable' + | 'in-flight' + | 'verify-policy-changed' + | 'stuck' + | 'merge-ready'; + detail: string; + command?: string; +} + export interface StatusReport { goalId: string; state: string; + base: string; + branch: string; + currentBranch: string; nextAction: NextAction; headSha: string; // current HEAD — chain this into the next record's expectedSha maxFixRounds: number; // the goal's fix budget — lets the progress card show "round N/max" @@ -16,6 +33,38 @@ export interface StatusReport { live: LiveGit; facts: DerivedFacts; warnings?: string[]; // non-blocking advisories (e.g. verify.testCommand changed in the goal diff) + warningLevel: 'none' | 'info' | 'warn' | 'block'; + edge?: StatusEdge; + progressKey: string; +} + +function edgeFor(goal: ReturnType, na: NextAction, branch: string, warnings: string[]): StatusEdge | undefined { + if (goal.inFlight) { + return { kind: 'in-flight', detail: `${goal.inFlight.kind} ${goal.inFlight.phase} in flight since ${goal.inFlight.startedAt}` }; + } + if (goal.reviewUnavailable) { + return { + kind: 'oracle-unavailable', + detail: `${goal.reviewUnavailable.phase} unavailable: ${goal.reviewUnavailable.reason} after ${goal.reviewUnavailable.attempts} attempt(s)`, + command: `coco goal op-clear --goal ${goal.id}`, + }; + } + if (na === 'wrong-branch') return { kind: 'wrong-branch', detail: `currently on ${branch}; goal branch is ${goal.branch}`, command: `git checkout ${goal.branch}` }; + if (na === 'commit-or-revert') return { kind: 'dirty-tree', detail: 'working tree is dirty; review/verify are blocked until committed or reverted' }; + if (na === 'rebase-needed') return { kind: 'rebase-needed', detail: `branch is behind ${goal.base}`, command: `git rebase ${goal.base}` }; + if (na === 'escalate-human') return { kind: 'stuck', detail: goal.failureLoop?.count ? `same failure repeated ${goal.failureLoop.count} time(s)` : 'fix budget or human blocker reached' }; + if (na === 'merge-gate') return { kind: 'merge-ready', detail: 'clean review + passing verify; awaiting human merge consent', command: `coco merge --goal ${goal.id}` }; + if (warnings.some((w) => w.includes('verify.testCommand differs'))) { + return { kind: 'verify-policy-changed', detail: 'verify.testCommand changed in this goal; merge requires explicit acknowledgement', command: `coco merge --goal ${goal.id} --ack-verify-policy-change` }; + } + return undefined; +} + +function warningLevel(na: NextAction, warnings: string[], edge?: StatusEdge): StatusReport['warningLevel'] { + if (na === 'escalate-human' || edge?.kind === 'oracle-unavailable' || edge?.kind === 'stuck') return 'block'; + if (edge?.kind === 'wrong-branch' || edge?.kind === 'dirty-tree' || edge?.kind === 'rebase-needed' || edge?.kind === 'verify-policy-changed') return 'warn'; + if (warnings.length) return 'info'; + return 'none'; } export function goalStatus(repo: string, id?: string): StatusReport { @@ -23,18 +72,29 @@ export function goalStatus(repo: string, id?: string): StatusReport { if (!goal) throw new Error('coco: no matching goal'); const live = gatherLive(repo, goal); + const branch = currentBranch(repo); + const facts = deriveFacts(goal.events, live.tHead); + const na = nextAction(goal, live); // Describe the goal diff even when status is read from another branch (use the goal branch then). const warnings = verifyConfigWarnings(repo, goal.base, live.onBranch ? 'HEAD' : goal.branch); + const edge = edgeFor(goal, na, branch, warnings); + const head = headSha(repo); return { goalId: goal.id, state: goal.state, - nextAction: nextAction(goal, live), - headSha: headSha(repo), + base: goal.base, + branch: goal.branch, + currentBranch: branch, + nextAction: na, + headSha: head, maxFixRounds: goal.maxFixRounds, ...(goal.backlogTaskId ? { backlogTaskId: goal.backlogTaskId } : {}), ...(goal.autoMergeAllowed ? { autoMergeAllowed: true } : {}), live, - facts: deriveFacts(goal.events, live.tHead), + facts, ...(warnings.length ? { warnings } : {}), + warningLevel: warningLevel(na, warnings, edge), + ...(edge ? { edge } : {}), + progressKey: `${goal.id}:${head.slice(0, 12)}:${na}:${edge?.kind ?? 'ok'}:${warnings.length}`, }; } From f6ed176377a04bbdfa6b508cdb95ec3ab15284bf Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:52:04 +0930 Subject: [PATCH 42/66] feat: enrich loop progress card for edge states --- src/progress/loop.ts | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/src/progress/loop.ts b/src/progress/loop.ts index 15d1c1c..c18c6e0 100644 --- a/src/progress/loop.ts +++ b/src/progress/loop.ts @@ -16,7 +16,7 @@ const CHECKPOINT: Record = { 'escalate-human': 'blocked — needs human', 'wrong-branch': 'recover — wrong branch', 'commit-or-revert': 'recover — dirty tree', - 'rebase-needed': 'rebase onto main', + 'rebase-needed': 'recover — rebase needed', none: 'inactive', }; @@ -32,7 +32,7 @@ const NEXT: Record = { 'escalate-human': 'escalate — surface the blocker to the human', 'wrong-branch': 'checkout the goal branch', 'commit-or-revert': 'commit or discard the working tree', - 'rebase-needed': 'rebase onto main', + 'rebase-needed': 'rebase onto the goal base', none: 'no active goal', }; @@ -47,7 +47,7 @@ const REMAINING: Record = { 'escalate-human': ['human decision'], 'wrong-branch': ['recover', '…'], 'commit-or-revert': ['recover', '…'], - 'rebase-needed': ['rebase', '…'], + 'rebase-needed': ['rebase', 'review', 'verify', 'merge'], none: [], }; @@ -61,7 +61,13 @@ function verifiedLine(r: StatusReport): string { const plan = stage(planDone, 'plan', na === 'plan'); const impl = stage(r.facts.implementAtEpoch, 'implement', na === 'implement'); const review = - r.facts.latestReview === 'clean' ? 'review ✓ clean' : r.facts.latestReview === 'blocking' ? 'review ✗ blocking' : 'review — pending'; + r.edge?.kind === 'oracle-unavailable' + ? 'review ? unavailable' + : r.facts.latestReview === 'clean' + ? 'review ✓ clean' + : r.facts.latestReview === 'blocking' + ? 'review ✗ blocking' + : 'review — pending'; const verify = r.facts.latestVerify === 'pass' ? 'verify ✓ pass' : r.facts.latestVerify === 'fail' ? 'verify ✗ fail' : 'verify — pending'; return `${plan} ${impl} ${review} ${verify}`; @@ -76,13 +82,29 @@ function checkpointLine(r: StatusReport): string { return base; } +function riskLine(r: StatusReport): string { + const warnings = r.warnings ?? []; + if (warnings.some((w) => w.includes('verify.testCommand differs'))) return 'verify policy changed — human acknowledgement required'; + if (warnings.length) return warnings.join(' · '); + return '—'; +} + +function recoveryLine(r: StatusReport): string { + if (r.edge?.command) return r.edge.command; + if (r.edge?.detail) return r.edge.detail; + return '—'; +} + export function loopView(r: StatusReport): ProgressView { const rows = [ { label: 'Checkpoint', value: checkpointLine(r) }, + { label: 'Branch', value: `${r.currentBranch} → ${r.base}${r.currentBranch === r.branch ? ' (on goal)' : ` · goal ${r.branch}`}` }, { label: 'Verified', value: verifiedLine(r) }, { label: 'Remaining', value: (REMAINING[r.nextAction] ?? []).join(' → ') || '—' }, + { label: 'Risk', value: riskLine(r) }, + { label: 'Recovery', value: recoveryLine(r) }, { label: 'Next', value: NEXT[r.nextAction] ?? String(r.nextAction) }, ]; - const provenance = `${r.goalId} · ${r.headSha.slice(0, 7)} · state=${r.state} · next=${r.nextAction}`; + const provenance = `${r.goalId} · ${r.headSha.slice(0, 7)} · state=${r.state} · next=${r.nextAction} · level=${r.warningLevel} · key=${r.progressKey}`; return { skill: 'coco-loop', subject: r.goalId, rows, provenance }; } From 09aea82e686c448bb6f6edec301ad0f30d7be15c Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:53:19 +0930 Subject: [PATCH 43/66] test: cover richer loop progress cards --- tests/progress.test.ts | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/tests/progress.test.ts b/tests/progress.test.ts index 0f53fd8..098a136 100644 --- a/tests/progress.test.ts +++ b/tests/progress.test.ts @@ -24,11 +24,16 @@ function report(over: Partial = {}): StatusReport { return { goalId: 'goal-20260707-0949-add-retry', state: 'active', + base: 'main', + branch: 'coco/goal-20260707-0949-add-retry', + currentBranch: 'coco/goal-20260707-0949-add-retry', nextAction: 'plan', headSha: 'a1b2c3d4e5f6', maxFixRounds: 5, live: { tHead: 't0', treeClean: true, onBranch: true, baseMerged: true }, facts: { implementAtEpoch: false, latestReview: 'none', latestVerify: 'none', fixRounds: 0 }, + warningLevel: 'none', + progressKey: 'k', ...over, }; } @@ -62,7 +67,7 @@ test('loopView renders every nextAction without throwing and stamps provenance', for (const na of ALL_ACTIONS) { const v = loopView(report({ nextAction: na })); expect(v.skill).toBe('coco-loop'); - expect(v.rows).toHaveLength(4); + expect(v.rows.length).toBeGreaterThanOrEqual(7); expect(v.provenance).toContain(`next=${na}`); // the whole thing renders to a fenced block cleanly expect(renderView(v)).toContain('◈ coco-loop'); @@ -82,14 +87,35 @@ test('loopView fix card shows the round and the blocking review, glyph + words', expect(byLabel.Next).toContain('fix'); }); -test('loopView verify-pass at merge-gate reads clean + pass', () => { +test('loopView verify-pass at merge-gate reads clean + pass and keeps command in recovery only', () => { const v = loopView( - report({ nextAction: 'merge-gate', facts: { implementAtEpoch: true, latestReview: 'clean', latestVerify: 'pass', fixRounds: 0 } }), + report({ + nextAction: 'merge-gate', + facts: { implementAtEpoch: true, latestReview: 'clean', latestVerify: 'pass', fixRounds: 0 }, + edge: { kind: 'merge-ready', detail: 'ready', command: 'coco merge --goal goal-20260707-0949-add-retry' }, + }), ); const byLabel = Object.fromEntries(v.rows.map((r) => [r.label, r.value])); expect(byLabel.Verified).toContain('review ✓ clean'); expect(byLabel.Verified).toContain('verify ✓ pass'); - expect(byLabel.Next).toContain('awaiting human approval'); // merge command stays OUT of the card + expect(byLabel.Next).toContain('awaiting human approval'); + expect(byLabel.Recovery).toContain('coco merge'); +}); + +test('loopView edge card shows wrong branch recovery command', () => { + const v = loopView( + report({ + nextAction: 'wrong-branch', + currentBranch: 'main', + edge: { kind: 'wrong-branch', detail: 'currently on main', command: 'git checkout coco/goal-20260707-0949-add-retry' }, + warningLevel: 'warn', + }), + ); + const byLabel = Object.fromEntries(v.rows.map((r) => [r.label, r.value])); + expect(byLabel.Checkpoint).toContain('wrong branch'); + expect(byLabel.Branch).toContain('main → main'); + expect(byLabel.Recovery).toBe('git checkout coco/goal-20260707-0949-add-retry'); + expect(v.provenance).toContain('level=warn'); }); test('storeView aggregates backlog counts, spec completion, and the latest roadmap line', () => { From 9ef71759b0c40fe02bf2cf1ab066c4da75ebad43 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:55:33 +0930 Subject: [PATCH 44/66] test: cover audit validation and feedback signals --- tests/audit.test.ts | 49 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 9 deletions(-) diff --git a/tests/audit.test.ts b/tests/audit.test.ts index 63bb586..b7aeb14 100644 --- a/tests/audit.test.ts +++ b/tests/audit.test.ts @@ -2,8 +2,8 @@ import { mkdtempSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { expect, test } from 'vitest'; -import { appendAudit, auditPath, buildAuditRecord, readAudit, type AuditRecord } from '../src/audit.js'; -import { auditReport } from '../src/commands/audit.js'; +import { appendAudit, appendAuditFeedback, auditPath, buildAuditRecord, readAudit, readAuditDetailed, type AuditRecord } from '../src/audit.js'; +import { auditReport, auditValidate } from '../src/commands/audit.js'; import { goalOracleUnavailable } from '../src/commands/goalOracle.js'; import { goalRecord } from '../src/commands/goalRecord.js'; import { goalStart } from '../src/commands/goalStart.js'; @@ -15,6 +15,9 @@ import { g, tmpRepo } from './helpers.js'; const goal = (over: Partial = {}): GoalState => ({ id: 'g1', state: 'active', events: [], updatedAt: '2026-07-07T00:00:00.000Z', ...over }) as unknown as GoalState; +const review = (verdict: 'clean' | 'blocking', tree: string): Partial => ({ phase: 'review', verdict, tree }); +const verify = (verdict: 'pass' | 'fail'): Partial => ({ phase: 'verify', verdict }); + test('buildAuditRecord: lean record for a non-event op, no phase/verdict', () => { const r = buildAuditRecord(goal(), 'goal-start'); expect(r).toEqual({ v: 1, at: '2026-07-07T00:00:00.000Z', goalId: 'g1', action: 'goal-start', state: 'active', events: 0 }); @@ -34,7 +37,7 @@ test('buildAuditRecord: carries the fingerprint fail counter when present', () = expect(r.failCount).toBe(3); }); -test('appendAudit + readAudit round-trip; readAudit skips torn lines', () => { +test('appendAudit + readAudit round-trip; readAudit skips torn lines while detailed read reports them', () => { const repo = mkdtempSync(join(tmpdir(), 'coco-audit-')); const rec: AuditRecord = { v: 1, at: '2026-07-07T00:00:00.000Z', goalId: 'g1', action: 'goal-start', state: 'active', events: 0 }; appendAudit(repo, rec); @@ -42,6 +45,9 @@ test('appendAudit + readAudit round-trip; readAudit skips torn lines', () => { const back = readAudit(repo); expect(back).toHaveLength(2); // the good record twice, the malformed line skipped expect(back[0]).toEqual(rec); + const detailed = readAuditDetailed(repo); + expect(detailed.invalid).toHaveLength(1); + expect(detailed.invalid[0].rawHash).toMatch(/^[0-9a-f]{16}$/); }); test('appendAudit is best-effort — never throws even when the path cannot be a directory', () => { @@ -50,25 +56,26 @@ test('appendAudit is best-effort — never throws even when the path cannot be a expect(() => appendAudit(notADir, buildAuditRecord(goal(), 'goal-start'))).not.toThrow(); }); -test('auditReport aggregates fix rounds, verify fails, oracle outages, and human-merge latency', () => { +test('auditReport aggregates fix rounds, verify fails, oracle outages, human-merge latency, and validity', () => { const repo = mkdtempSync(join(tmpdir(), 'coco-audit-')); const put = (goalId: string, action: string, at: string, extra: Partial = {}) => appendAudit(repo, { v: 1, at, goalId, action, state: extra.state ?? 'active', events: extra.events ?? 0, ...extra }); // goal A: one blocking round, then clean → verify pass → merged 30s later put('gA', 'goal-start', '2026-07-07T09:59:00.000Z'); - put('gA', 'record:review:blocking', '2026-07-07T09:59:10.000Z', { tree: 't1' }); - put('gA', 'record:review:clean', '2026-07-07T09:59:50.000Z', { tree: 't2' }); - put('gA', 'record:verify:pass', '2026-07-07T10:00:00.000Z'); + put('gA', 'record:review:blocking', '2026-07-07T09:59:10.000Z', review('blocking', 't1')); + put('gA', 'record:review:clean', '2026-07-07T09:59:50.000Z', review('clean', 't2')); + put('gA', 'record:verify:pass', '2026-07-07T10:00:00.000Z', verify('pass')); put('gA', 'merge', '2026-07-07T10:00:30.000Z', { state: 'achieved' }); // goal B: a verify fail and an Oracle outage, never merged put('gB', 'goal-start', '2026-07-07T11:00:00.000Z'); - put('gB', 'record:verify:fail', '2026-07-07T11:01:00.000Z'); + put('gB', 'record:verify:fail', '2026-07-07T11:01:00.000Z', verify('fail')); put('gB', 'oracle-unavailable:review:oracle-timeout', '2026-07-07T11:02:00.000Z'); const rep = auditReport(repo); expect(rep.totalRecords).toBe(8); expect(rep.totals).toEqual({ goals: 2, fixRounds: 1, verifyFails: 1, oracleOutages: 1, merges: 1 }); + expect(rep.validity.ok).toBe(true); const a = rep.goals.find((x) => x.goalId === 'gA')!; expect(a).toMatchObject({ fixRounds: 1, verifyFails: 0, oracleOutages: 0, reachedMerge: true, finalState: 'achieved', verifyToMergeSec: 30 }); @@ -79,7 +86,7 @@ test('auditReport aggregates fix rounds, verify fails, oracle outages, and human test('auditReport fixRounds counts distinct blocking trees, not blocking-review records', () => { const repo = mkdtempSync(join(tmpdir(), 'coco-audit-')); const put = (action: string, at: string, tree?: string) => - appendAudit(repo, { v: 1, at, goalId: 'g', action, state: 'active', events: 0, ...(tree ? { tree } : {}) }); + appendAudit(repo, { v: 1, at, goalId: 'g', action, state: 'active', events: 0, ...(tree ? review('blocking', tree) : {}) }); put('goal-start', '2026-07-07T00:00:00.000Z'); put('record:review:blocking', '2026-07-07T00:01:00.000Z', 'tA'); put('record:review:blocking', '2026-07-07T00:02:00.000Z', 'tA'); // same tree — a re-review, NOT a new fix round @@ -87,6 +94,30 @@ test('auditReport fixRounds counts distinct blocking trees, not blocking-review expect(auditReport(repo).goals[0].fixRounds).toBe(2); // tA, tB — not 3 }); +test('auditValidate catches invalid lines and impossible merge ordering', () => { + const repo = mkdtempSync(join(tmpdir(), 'coco-audit-')); + appendAudit(repo, { v: 1, at: '2026-07-07T00:00:00.000Z', goalId: 'g', action: 'goal-start', state: 'active', events: 0 }); + appendAudit(repo, { v: 1, at: '2026-07-07T00:01:00.000Z', goalId: 'g', action: 'merge', state: 'achieved', events: 0 }); + writeFileSync(auditPath(repo), '{ bad json\n', { flag: 'a' }); + const v = auditValidate(repo); + expect(v.ok).toBe(false); + expect(v.invalidRecords).toBe(1); + expect(v.failures.map((f) => f.code)).toContain('merge-before-verify-pass'); +}); + +test('structured feedback is redacted and summarized', () => { + const repo = mkdtempSync(join(tmpdir(), 'coco-audit-')); + appendAuditFeedback(repo, { goalId: 'g1', kind: 'goal-quality', rating: 2, tags: ['Vague Goal', 'vague goal'], note: 'This raw note must not be stored.' }); + appendAuditFeedback(repo, { goalId: 'g2', kind: 'goal-quality', rating: 4, tags: ['clear'] }); + const rec = readAudit(repo).find((r) => r.action === 'feedback:goal-quality')!; + expect(JSON.stringify(rec)).not.toContain('raw note'); + expect(rec.noteHash).toMatch(/^[0-9a-f]{16}$/); + const feedback = auditReport(repo).feedback; + expect(feedback.total).toBe(2); + expect(feedback.byKind['goal-quality'].avgRating).toBe(3); + expect(feedback.topTags[0]).toEqual(['vague goal', 1]); +}); + test('goalOracleUnavailable is captured AND its evidence is centrally capped', () => { const repo = tmpRepo(); initRepo(repo); From 66fc7a984095f03f20fd7c8a13caac8856298f53 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:57:15 +0930 Subject: [PATCH 45/66] test: keep doctor fixtures valid under goal schema --- tests/doctor.test.ts | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/tests/doctor.test.ts b/tests/doctor.test.ts index 6bdf890..da2fad2 100644 --- a/tests/doctor.test.ts +++ b/tests/doctor.test.ts @@ -7,6 +7,7 @@ import { goalStart } from '../src/commands/goalStart.js'; import { initRepo } from '../src/commands/init.js'; import { isGuardInstalled } from '../src/commands/installHooks.js'; import { goalsDir } from '../src/paths.js'; +import type { GoalState } from '../src/types.js'; import { g, tmpRepo } from './helpers.js'; function initedRepo(): string { @@ -19,6 +20,10 @@ function initedRepo(): string { // An EMPTY home so the wiring checks read no real ~/.codex/~/.claude config. const emptyHome = (): string => mkdtempSync(join(tmpdir(), 'coco-home-')); +function goalFixture(id: string, state: GoalState['state']): GoalState { + return { id, objective: id, branch: `coco/${id}`, base: 'main', state, maxFixRounds: 3, acceptanceChecks: [], events: [] }; +} + function mkRun(repo: string, runId: string, goalId?: string): string { const dir = join(repo, '.coco', 'verify-runs', runId); mkdirSync(dir, { recursive: true }); @@ -32,13 +37,14 @@ test('runDoctor produces a grouped report with a consistent summary', () => { const rep = runDoctor(repo, { home: emptyHome() }); const names = rep.checks.map((c) => c.name); - expect(names).toEqual(expect.arrayContaining(['node', 'git', 'git repo', 'coco initialized', 'verify.testCommand', 'oracle MCP', 'active goal'])); + expect(names).toEqual(expect.arrayContaining(['node', 'git', 'git repo', 'coco initialized', 'verify.testCommand', 'oracle MCP', 'active goal', 'audit validity'])); expect(rep.summary.ok + rep.summary.warn + rep.summary.fail).toBe(rep.checks.length); // every check tallied const by = (n: string) => rep.checks.find((c) => c.name === n)!; expect(by('node').status).toBe('ok'); expect(by('git repo').status).toBe('ok'); expect(by('coco initialized').status).toBe('ok'); + expect(by('audit validity').status).toBe('ok'); expect(by('active goal').detail).toBe('no active goal'); // fresh init, none active expect(by('oracle MCP').status).toBe('warn'); // empty home → not registered }); @@ -57,8 +63,8 @@ test('cleanDoctor: reclaims terminal + orphaned runs only; preserves active/bloc const repo = initedRepo(); const { goalId } = goalStart(repo, { objective: 'doctor clean', maxFixRounds: 5, acceptanceChecks: [] }); // sibling goal files in terminal + blocked lifecycle states - writeFileSync(join(goalsDir(repo), 'goal-term.json'), JSON.stringify({ id: 'goal-term', state: 'achieved', events: [] })); - writeFileSync(join(goalsDir(repo), 'goal-block.json'), JSON.stringify({ id: 'goal-block', state: 'blocked', events: [] })); + writeFileSync(join(goalsDir(repo), 'goal-term.json'), JSON.stringify(goalFixture('goal-term', 'achieved'))); + writeFileSync(join(goalsDir(repo), 'goal-block.json'), JSON.stringify(goalFixture('goal-block', 'blocked'))); const activeRun = mkRun(repo, 'run-active', goalId); // active → preserve const blockedRun = mkRun(repo, 'run-blocked', 'goal-block'); // blocked/resumable → preserve @@ -68,7 +74,7 @@ test('cleanDoctor: reclaims terminal + orphaned runs only; preserves active/bloc const dry = cleanDoctor(repo); expect(dry.applied).toBe(false); - expect(dry.reclaimedBytes).toBe(0); + expect(dry.reclaimedBytes).toBeGreaterThan(0); expect(dry.targets.map((t) => t.path).sort()).toEqual([orphanRun, terminalRun].sort()); expect(existsSync(terminalRun)).toBe(true); // dry-run deletes nothing From f6cadd7f5ae88b888ddb09d593f41362c50cefe3 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:58:01 +0930 Subject: [PATCH 46/66] docs: register audit learning commands --- src/commands/registry.ts | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/commands/registry.ts b/src/commands/registry.ts index 1a422e8..52a499a 100644 --- a/src/commands/registry.ts +++ b/src/commands/registry.ts @@ -17,9 +17,12 @@ const COMMANDS: readonly CommandDescriptor[] = [ { name: 'coco merge', surfaces: ['cli', 'skill'], effect: 'destructive', summary: 'Human-terminal FF-only merge path.', safety: 'Requires clean review, passing verify, current epoch, clean branch, base ancestry, and explicit ack for verify policy changes.' }, { name: 'coco_merge', surfaces: ['mcp', 'skill'], effect: 'destructive', summary: 'Opt-in auto-merge attempt for a single goal.', safety: 'Requires per-goal consent and risk-tier; falls back to human on risk.' }, { name: 'coco health', surfaces: ['cli', 'mcp'], effect: 'read', summary: 'Report loop health, stall, conflict, in-flight, and invalid-state conditions.' }, - { name: 'coco doctor', surfaces: ['cli'], effect: 'read', summary: 'Aggregate local environment, repo, wiring, goal, and data hygiene checks.' }, + { name: 'coco doctor', surfaces: ['cli'], effect: 'read', summary: 'Aggregate local environment, repo, wiring, goal, audit, and data hygiene checks.' }, { name: 'coco doctor clean', surfaces: ['cli'], effect: 'destructive', summary: 'Dry-run/apply cleanup of terminal or orphaned verify-run cache only.' }, - { name: 'coco improve digest', surfaces: ['cli', 'skill'], effect: 'read', summary: 'Summarise structural audit signals with an insufficient-data gate.' }, + { name: 'coco audit report', surfaces: ['cli'], effect: 'read', summary: 'Summarise valid audit records, feedback, and loop trajectory signals.' }, + { name: 'coco audit validate', surfaces: ['cli'], effect: 'read', summary: 'Validate audit schema and cross-record invariants before self-improvement acts.' }, + { name: 'coco audit feedback', surfaces: ['cli'], effect: 'write', summary: 'Append structured redacted human feedback for goal/implementation/loop quality.' }, + { name: 'coco improve digest', surfaces: ['cli', 'skill'], effect: 'read', summary: 'Summarise structural audit + feedback signals with an insufficient-data gate.' }, { name: 'coco improve check', surfaces: ['cli', 'skill'], effect: 'read', summary: 'Refuse protected-path targets for self-improvement proposals.' }, { name: 'coco improve promote', surfaces: ['cli', 'skill'], effect: 'write', summary: 'Promote one non-protected improve task linked to a local improve-spec.' }, { name: 'coco eval', surfaces: ['cli'], effect: 'read', summary: 'Run deterministic safety-regression fixture checks.' }, From 04683dbf57851058136bba388a53ea1ef1f68c05 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:59:02 +0930 Subject: [PATCH 47/66] feat: add audit validity eval fixture --- src/commands/eval.ts | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/src/commands/eval.ts b/src/commands/eval.ts index ffadad3..ac8705d 100644 --- a/src/commands/eval.ts +++ b/src/commands/eval.ts @@ -1,12 +1,17 @@ +import { mkdtempSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { appendAudit, auditPath } from '../audit.js'; import { deriveFacts } from '../epoch.js'; import { FINGERPRINT_N } from '../fingerprint.js'; import { nextAction } from '../gate.js'; import { parseOracleVerdict } from '../oracleVerdict.js'; import type { GoalEvent, GoalState } from '../types.js'; +import { auditValidate } from './audit.js'; export interface EvalCaseResult { id: string; - area: 'verdict' | 'epoch' | 'gate' | 'privacy' | 'self-improve'; + area: 'verdict' | 'epoch' | 'gate' | 'audit' | 'privacy' | 'self-improve'; invariant: string; ok: boolean; detail?: string; @@ -44,6 +49,15 @@ function activeGoal(partial: Partial = {}): GoalState { }; } +function auditInvalidFixture(): boolean { + const repo = mkdtempSync(join(tmpdir(), 'coco-eval-audit-')); + appendAudit(repo, { v: 1, at: '2026-07-08T00:00:00.000Z', goalId: 'g', action: 'goal-start', state: 'active', events: 0 }); + appendAudit(repo, { v: 1, at: '2026-07-08T00:01:00.000Z', goalId: 'g', action: 'merge', state: 'achieved', events: 0 }); + writeFileSync(auditPath(repo), '{bad json\n', { flag: 'a' }); + const v = auditValidate(repo); + return !v.ok && v.invalidRecords === 1 && v.failures.some((f) => f.code === 'merge-before-verify-pass'); +} + const cases: EvalCase[] = [ { id: 'oracle-verdict-ambiguous-fails-closed', @@ -94,6 +108,12 @@ const cases: EvalCase[] = [ { tHead: 't', treeClean: false, onBranch: true, baseMerged: true }, ) === 'escalate-human', }, + { + id: 'audit-invalidity-is-detected', + area: 'audit', + invariant: 'Invalid/torn audit lines and impossible merge ordering are detected before self-improvement trusts them.', + run: auditInvalidFixture, + }, ]; export function runEval(): EvalReport { From 3df135053936442de41edfaedff30f61078232c8 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:00:00 +0930 Subject: [PATCH 48/66] docs: document audit-backed self-evolution loop --- docs/self-evolution.md | 97 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 docs/self-evolution.md diff --git a/docs/self-evolution.md b/docs/self-evolution.md new file mode 100644 index 0000000..bb2d048 --- /dev/null +++ b/docs/self-evolution.md @@ -0,0 +1,97 @@ +# Audit-backed self-evolution + +coco should improve because its **signals get better**, not because it edits itself more often. The self-evolution loop is deliberately conservative: + +1. capture local, structured audit records; +2. validate the audit substrate; +3. add structured human feedback where the audit cannot infer quality; +4. digest signals deterministically; +5. propose one small, non-protected improvement; +6. attach or add an eval/non-regression check; +7. run the normal coco-loop review/verify/human-merge gate. + +## Audit validity comes first + +`coco audit validate` checks whether the local `.coco/audit.ndjson` stream is trustworthy enough to use as an improvement substrate. It reports torn/invalid lines and cross-record invariant failures such as merge records appearing before verify passes. + +```sh +coco audit validate +``` + +`coco doctor` also surfaces audit validity as a data check. If audit validity fails, self-improvement should pause until a human fixes the audit substrate. A system that cannot trust its measurements must not rewrite its own workflow. + +## Structured feedback + +Some quality signals cannot be inferred from structural telemetry. Add lightweight human feedback after a loop: + +```sh +coco audit feedback \ + --goal \ + --kind goal-quality \ + --rating 2 \ + --tags vague-goal,weak-verification +``` + +Feedback kinds: + +- `goal-quality` — the goal/spec was vague, too broad, missing proof, or well-scoped. +- `implementation-quality` — the implementation was incomplete, overbroad, clean, or robust. +- `loop-friction` — the workflow was annoying, slow, ambiguous, or smooth. +- `review-quality` — Oracle review found useful issues, missed issues, or generated noise. +- `verification-quality` — tests/verify were trustworthy or weak. +- `status-clarity` — Codex.app / Claude Code progress and recovery guidance was clear or confusing. + +Feedback is redacted: tags and ratings are stored, and optional notes are hashed with length metadata. Raw note text is not written to the audit stream. + +## Improve digest + +`coco improve digest` reads the validated audit report and emits deterministic signals: + +- Oracle reliability problems. +- Goal-quality feedback patterns. +- Status-clarity feedback patterns. +- Implementation-quality diagnostics. +- Audit-validity problems. +- Churn, verify failure, and merge-latency observations. + +Only fired signals marked `safeToActOn:true` may become a `$coco-improve` proposal, and even then the proposal must pass protected-path checks and include an eval or explicit non-regression check. + +## No eval, no improvement + +A self-improvement proposal should answer: + +- Which audit/feedback signal fired? +- What is the predeclared hypothesis? +- What mechanism should improve signal quality? +- What failure would disprove it? +- Which eval or regression test protects the behavior? + +Use `coco eval` for deterministic safety fixtures and add new cases for recurring failure modes. + +```sh +coco eval +``` + +A good self-improvement PR makes future loops safer or clearer while preserving all referee invariants. A bad one merely reduces friction by weakening scrutiny. + +## What not to optimize + +Never optimize these as primary success metrics: + +- fewer review rounds; +- fewer fix rounds; +- faster time to merge; +- higher verify pass rate; +- more auto-merges. + +These can indicate improvement only when paired with stronger signal quality and no increase in false-greens. + +## Daily practice + +After using coco for a real goal: + +1. Run `coco audit report` to inspect the structural trace. +2. Add feedback if the goal or loop quality was materially good or bad. +3. Let `$coco-improve` act only after the sample gate passes. +4. Review the proposed spec/task as a hypothesis, not proof. +5. Run the task through `$coco-loop` like any other change. From 01d4c7ed5a1f122641116693526d9971ed08ca5e Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:01:26 +0930 Subject: [PATCH 49/66] docs: require audit validity and eval-backed improvement --- skills/coco-improve/SKILL.md | 51 +++++++++++++++++++----------------- 1 file changed, 27 insertions(+), 24 deletions(-) diff --git a/skills/coco-improve/SKILL.md b/skills/coco-improve/SKILL.md index de786a9..17cf5cf 100644 --- a/skills/coco-improve/SKILL.md +++ b/skills/coco-improve/SKILL.md @@ -1,51 +1,54 @@ --- name: coco-improve -description: Use when the user types $coco-improve or /coco-improve, or asks to analyse coco's own loop history and propose an improvement to the coco skills/CLI. The self-improvement layer — reflect over the audit corpus (plus bounded, cited web research on the signal's topic), form ONE incremental, evidence-backed hypothesis, archive it as a LOCAL improve-spec card, promote ONE non-protected loop-sized task, then STOP. Propose-only: never builds, never merges, never starts coco-loop, never edits live skills, never touches the referee. +description: Use when the user types $coco-improve or /coco-improve, or asks to analyse coco's own loop history and propose an improvement to the coco skills/CLI. The self-improvement layer — validate the audit substrate, reflect over audit + structured feedback (plus bounded, cited web research on the signal's topic), form ONE incremental eval-backed hypothesis, archive it as a LOCAL improve-spec card, promote ONE non-protected loop-sized task, then STOP. Propose-only: never builds, never merges, never starts coco-loop, never edits live skills, never touches the referee. --- # coco-improve -Turn coco's own **audit history** into ONE **incremental, evidence-backed improvement proposal** for the coco skills/CLI — archived as a local spec card and promoted as ONE loop-sized backlog task for a human to review. You **reflect and propose** (a spec + one non-protected task); the **human** decides whether to run it; **coco-loop** builds it under the usual Oracle-review + coco-verify + human-merge gate. You never build, never merge, never edit live skills, and never touch the referee. +Turn coco's own **valid audit history + structured human feedback** into ONE **incremental, eval-backed improvement proposal** for the coco skills/CLI — archived as a local spec card and promoted as ONE loop-sized backlog task for a human to review. You **reflect and propose** (a spec + one non-protected task); the **human** decides whether to run it; **coco-loop** builds it under the usual Oracle-review + coco-verify + human-merge gate. You never build, never merge, never edit live skills, and never touch the referee. Invoke: `$coco-improve` (Codex) / `/coco-improve` (Claude Code). ## Golden rules (do not deviate) 1. **Propose-only.** Archive ONE local improve-spec card AND promote ONE loop-sized, non-protected task linked to it (via `coco improve promote`, which enforces the protected-path guard in code), then STOP. Do NOT start coco-loop, do NOT build or merge, do NOT edit any `skills/**` or source file directly. Building + merging stay with coco-loop + the human. -2. **Insufficient-data gate.** Start with `coco improve digest`. If `sufficient:false`, STOP and tell the user there isn't enough audit history yet — never manufacture a proposal from a thin window. -3. **Act only on a live, safe signal.** Pick ONE signal with `status:"signal"` **and** `safeToActOn:true` (today only `oracle-reliability`). If none qualifies, STOP — report the digest's diagnostics and hand back. `recurring-churn`, `verify-failures`, and `human-merge-latency` are **diagnostic/observational only, NEVER optimisation targets** (optimising them rewards weaker scrutiny). -4. **Protected-path gate (hard, not advisory).** Before proposing, run `coco improve check ` on the exact files the change would touch. If it returns `ok:false` (referee / metrics / CLI-MCP surface / evaluator / improve-self / runtime), STOP — that needs a **human-authored referee-change goal OUTSIDE coco-improve**. Never route a referee change through here. -5. **Incremental deltas only.** Propose the smallest targeted change (e.g. add a retry/resume clause to the loop skill) — NEVER a wholesale `SKILL.md` rewrite (that collapses hard-won context). -6. **Carry the anti-goals.** Copy the digest's `antiGoals` verbatim into the spec's `Anti-goals` section. The target is **signal quality** (fewer false-greens / stalls / Oracle outages), never throughput. An oracle-reliability fix must **not** loosen retry-once or verdict strictness. -7. **Local + redacted.** Archive the spec `--visibility local` so audit-derived content never travels to Oracle. Keep rejected alternatives inside the spec (or as local cards) — never as backlog churn. +2. **Audit-validity gate.** Start with `coco audit validate`. If `ok:false`, STOP and tell the user the audit substrate is not trustworthy enough for self-improvement. Do not act on invalid/torn/impossible telemetry. +3. **Insufficient-data gate.** Then run `coco improve digest`. If `sufficient:false`, STOP and tell the user there isn't enough audit history yet — never manufacture a proposal from a thin window. +4. **Act only on a live, safe signal.** Pick ONE signal with `status:"signal"` **and** `safeToActOn:true` (e.g. `oracle-reliability`, `goal-quality-feedback`, `status-clarity-feedback`). If none qualifies, STOP — report diagnostics and hand back. `recurring-churn`, `implementation-quality-feedback`, `verify-failures`, `human-merge-latency`, and `audit-validity` are diagnostic/observational unless code marks them safe. +5. **No eval, no improvement.** Every proposal must name an existing `coco eval` fixture it improves/protects OR add a new deterministic test/eval in the task body. Do not promote a task that cannot be falsified. +6. **Protected-path gate (hard, not advisory).** Before proposing, run `coco improve check ` on the exact files the change would touch. If it returns `ok:false` (referee / metrics / CLI-MCP surface / evaluator / improve-self / runtime), STOP — that needs a **human-authored referee-change goal OUTSIDE coco-improve**. Never route a referee change through here. +7. **Incremental deltas only.** Propose the smallest targeted change (e.g. add a retry/resume clause to the loop skill or improve a progress card) — NEVER a wholesale `SKILL.md` rewrite. +8. **Carry the anti-goals.** Copy the digest's `antiGoals` verbatim into the spec's `Anti-goals` section. The target is **signal quality** (fewer false-greens / stalls / Oracle outages / unclear statuses), never throughput. An oracle-reliability fix must **not** loosen retry-once or verdict strictness. +9. **Local + redacted.** Archive the spec `--visibility local` so audit/feedback-derived content never travels to Oracle. Keep rejected alternatives inside the spec (or as local cards) — never as backlog churn. ## The loop -1. **Digest.** Run `coco improve digest`. Read `sufficient`, `signals`, `antiGoals`. `sufficient:false` → STOP (rule 2). -2. **Pick a target signal.** The first signal with `status:"signal"` and `safeToActOn:true`. None → STOP (rule 3), reporting the diagnostics. -3. **Reflect + bounded research.** +1. **Validate.** Run `coco audit validate`. `ok:false` → STOP (rule 2). Surface the failure codes, not raw audit lines. +2. **Digest.** Run `coco improve digest`. Read `sufficient`, `signals`, `antiGoals`, and feedback/audit counts. `sufficient:false` → STOP (rule 3). +3. **Pick a target signal.** The first signal with `status:"signal"` and `safeToActOn:true`. None → STOP (rule 4), reporting diagnostics. +4. **Reflect + bounded research.** a. **Read the repo** (reuse **Explore** / grep — never build an index) to locate the *smallest* incremental change that addresses the signal's cause. - b. **Research the topic — privacy-critical.** A fired safe signal carries a code-defined `researchTopic`. Search **exactly that string** (≤2 searches, ≤3 sources; WebSearch / deep-research / context7 / ferris-search). **NEVER** put coco's state into a query — no repo name, file paths, ids, counts, timestamps, or audit `detail` — and **never send the digest, a spec, audit records, or local filenames to Oracle or any external model during research** (the query is the generic `researchTopic`, nothing more). No `researchTopic` on the signal → skip research. - c. **Form ONE hypothesis, with typed evidence.** An external claim needs a **cited resolvable URL**; a local claim needs **repo/audit evidence**; your own reasoning may appear only as a **labeled hypothesis** tied to one of those. A cited best-practice is **evidence to test, never a mandate** — state *why* it applies to coco's specific mechanism, not merely that it's recommended (guard against authority bias). Note 1–2 rejected alternatives and why. -4. **Guard.** Run `coco improve check `. `ok:false` → STOP (rule 4). A proposal may only touch **non-protected** files (skills/docs/non-referee code). -5. **Author the improve-spec.** Write a spec body with **all** of these sections (coco-store REJECTS an improve spec missing any): `Outcome`, `Verification surface`, `Boundaries`, `Predeclared hypothesis`, `Audit evidence window`, `Expected mechanism`, `Success criteria`, `Failure criteria`, `Confounders`, `Rejected alternatives`, `Anti-goals`, `Research provenance`. Ground `Audit evidence window` in the digest numbers (`window.goals`, the signal's `sample` + `detail`) — never in a subjective read. `Research provenance` lists each cited external source (resolvable URL) with a one-line note on *why it applies to coco's mechanism* — or exactly `none - audit-only` if no external evidence was used. -6. **Archive (local).** + b. **Research the topic — privacy-critical.** A fired safe signal carries a code-defined `researchTopic`. Search **exactly that string** (≤2 searches, ≤3 sources; WebSearch / deep-research / context7 / ferris-search). **NEVER** put coco's state into a query — no repo name, file paths, ids, counts, timestamps, or audit `detail` — and **never send the digest, a spec, audit records, feedback, or local filenames to Oracle or any external model during research**. No `researchTopic` → skip research. + c. **Form ONE hypothesis, with typed evidence.** An external claim needs a **cited resolvable URL**; a local claim needs **repo/audit/feedback evidence**; your own reasoning may appear only as a **labeled hypothesis** tied to one of those. A cited best-practice is **evidence to test, never a mandate**. +5. **Guard.** Run `coco improve check `. `ok:false` → STOP (rule 6). A proposal may only touch **non-protected** files (skills/docs/non-referee code). +6. **Predeclare eval/non-regression.** Identify the exact existing `coco eval` case or test to protect the hypothesis, or state the new eval/test the task must add. This must be in the spec and task body. +7. **Author the improve-spec.** Write a spec body with **all** of these sections (coco-store REJECTS an improve spec missing any): `Outcome`, `Verification surface`, `Boundaries`, `Predeclared hypothesis`, `Audit evidence window`, `Expected mechanism`, `Success criteria`, `Failure criteria`, `Confounders`, `Rejected alternatives`, `Anti-goals`, `Research provenance`. Ground `Audit evidence window` in digest numbers (`window.goals`, `window.feedback`, `window.invalidAudit`, the signal's `sample` + `detail`) — never in a subjective read. `Verification surface` must name the eval/test. +8. **Archive (local).** ```sh coco-store add --type spec --title "improve: " --tags coco-improve --visibility local ``` Capture the returned `id` as `SPEC_ID`. -7. **Promote ONE task (code-guarded).** Promote the single loop-sized change linked to the spec. `coco improve promote` runs the protected-path check on `--paths` **in code** and REFUSES (exit 3) if any target is protected: +9. **Promote ONE task (code-guarded).** Promote the single loop-sized change linked to the spec. `coco improve promote` runs the protected-path check on `--paths` **in code** and REFUSES (exit 3) if any target is protected: ```sh coco improve promote --spec "$SPEC_ID" --id "improve-" --title "improve: " \ - --paths "" --body "" --priority medium + --paths "" --body "" --priority medium ``` - `--paths` are the exact files the change will touch (skills/docs/non-referee code). If it refuses, the target is protected → STOP (rule 4): that needs a human-authored referee-change goal outside coco-improve. -8. **Stop and hand off.** - > Improve-spec `` archived (local) and task `improve-` promoted to the backlog, evidenced by ``. Your call — review it, and if you agree, run `$coco-loop` to build it under the usual Oracle-review + coco-verify + human-merge gate. coco-improve does not build or merge. +10. **Stop and hand off.** + > Improve-spec `` archived (local) and task `improve-` promoted to the backlog, evidenced by ``. Your call — review it, and if you agree, run `$coco-loop` to build it under the usual Oracle-review + coco-verify + human-merge gate. coco-improve does not build or merge. ## Notes -- **Research is bounded, cited, and private.** External research is the signal's `researchTopic` only — capped (≤2 searches / ≤3 sources), and **nothing about coco leaves the machine in a query** (no state in the query, no digest/spec/audit sent to Oracle or any external model). Findings are evidence to **test** under the same human + coco-loop gate; a cited "best practice" is never a mandate and must be justified against coco's local mechanism (`Research provenance`). +- **Research is bounded, cited, and private.** External research is the signal's `researchTopic` only — capped (≤2 searches / ≤3 sources), and **nothing about coco leaves the machine in a query** (no state in the query, no digest/spec/audit/feedback sent to Oracle or any external model). - **One proposal per run.** A single well-evidenced, incremental change a human can judge — not a batch. -- **The referee is sacred.** `coco improve check` is the code-owned boundary; rule 4 is not negotiable. If you find yourself wanting to touch `src/gate.ts`, verdict parsing, verify, merge/risk, the metrics, the MCP/CLI surface, the spec evaluator, or coco-improve's own files — STOP and tell the human it needs a referee-change goal outside coco-improve. Even if a promoted task under-declares its `--paths`, an improve-origin goal whose **actual diff** touches a protected path is **refused at merge** (code-enforced) — a referee change simply cannot ride in through coco-improve. -- **Low-N honesty.** The digest's sample gate and `safeToActOn` flag exist because loop outcomes are high-variance; treat a proposal as *evidence to test*, not proof. The spec's Success/Failure criteria are how a later human judges it — do not pre-declare victory. +- **The referee is sacred.** `coco improve check` is the code-owned boundary; rule 6 is not negotiable. If you find yourself wanting to touch `src/gate.ts`, verdict parsing, verify, merge/risk, the metrics, the MCP/CLI surface, the spec evaluator, or coco-improve's own files — STOP and tell the human it needs a referee-change goal outside coco-improve. +- **Low-N honesty.** The digest's sample gates and `safeToActOn` flag exist because loop outcomes are high-variance; treat a proposal as *evidence to test*, not proof. The spec's Success/Failure criteria are how a later human judges it — do not pre-declare victory. From e91c320ce6d6c9d199465358bb4b01ebfc7394fe Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:04:08 +0930 Subject: [PATCH 50/66] fix: preserve feedback tag recency on tie --- src/commands/audit.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/commands/audit.ts b/src/commands/audit.ts index d9f0d27..ae5cf21 100644 --- a/src/commands/audit.ts +++ b/src/commands/audit.ts @@ -1,4 +1,4 @@ -import { readAuditDetailed, type AuditFeedbackKind, type AuditRecord } from '../audit.js'; +import { readAuditDetailed, type AuditRecord } from '../audit.js'; // Deterministic analysis over the captured audit stream (no LLM). Surfaces the signals coco-improve // reflects on: where the loop churns (fix rounds), where it fails (verify), where Oracle falls over @@ -113,7 +113,7 @@ function feedbackSummary(recs: AuditRecord[]): FeedbackSummary { negative: feedback.filter((r) => (r.rating ?? 5) <= 2).length, ...(a !== undefined ? { avgRating: a } : {}), byKind, - topTags: [...tags.entries()].sort((x, y) => y[1] - x[1] || x[0].localeCompare(y[0])).slice(0, 10), + topTags: [...tags.entries()].sort((x, y) => y[1] - x[1]).slice(0, 10), }; } From bf08b0d675cecb508ab85c3fb2ad9e1e5b6b2d4c Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:22:36 +0930 Subject: [PATCH 51/66] design: add cuter comic coco icon --- assets/coco-icon.svg | 106 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 assets/coco-icon.svg diff --git a/assets/coco-icon.svg b/assets/coco-icon.svg new file mode 100644 index 0000000..b8193e2 --- /dev/null +++ b/assets/coco-icon.svg @@ -0,0 +1,106 @@ + + coco comic icon + A cute comic-style coconut mascot with a speech bubble saying coco. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + coco + coco + coco + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From 458e3f4520d86834e95ac648582a044c8152b296 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:24:56 +0930 Subject: [PATCH 52/66] docs: use comic SVG coco icon --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3332b11..32942f5 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@

- coco + coco

# coco From 31adff71c3c7335840f15edd9b5acf6be3875a8e Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:49:04 +0930 Subject: [PATCH 53/66] feat: add coco-queue skill --- skills/coco-queue/SKILL.md | 51 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 skills/coco-queue/SKILL.md diff --git a/skills/coco-queue/SKILL.md b/skills/coco-queue/SKILL.md new file mode 100644 index 0000000..c42bfc1 --- /dev/null +++ b/skills/coco-queue/SKILL.md @@ -0,0 +1,51 @@ +--- +name: coco-queue +description: Use when the user types $coco-queue or /coco-queue, asks “what should coco do next?”, “next task”, “show queue”, “pick from backlog”, or wants the next ready project task before starting a loop. Queue triage skill for coco — reads BACKLOG/coco-store status, surfaces the next ready task, explains why it is next, and stops. Never implements or merges. +--- + +# coco-queue + +Pick or explain the **next ready project task** without implementing it. This is the queue/PM handoff layer between `$coco-goal` and `$coco-loop`. + +Invoke: `$coco-queue` / `/coco-queue`, optionally with `next`, `status`, `why`, or a filter like `docs`, `tests`, `low-risk`. + +## Golden rules + +1. **Read-only by default.** Do not edit files, start a goal, run `$coco-loop`, or merge. +2. **Use the code-owned queue first.** Call `coco_next({ repoDir })` / `coco next` before inventing work. +3. **Show project state.** Also use `coco-store progress` or `coco-store status` when available so the user can see backlog/spec context. +4. **Do not fabricate queue items.** If no ready task exists, say so and suggest `$coco-goal ` or `$coco-night --plan-next` only if the user wants coco to create a task. +5. **Prefer small, verified work.** If several tasks appear eligible, prefer tasks with a linked spec, clear verification surface, no blocked dependencies, and low blast radius. +6. **Stop at the handoff.** Output the recommended next task and the exact next command; do not run it. + +## Workflow + +1. Resolve `repoDir` to the current repository root. +2. Run `coco_next({ repoDir })` / `coco next`. +3. Run `coco-store progress` or `coco-store status` if available. +4. If a task exists, present: + - task id; + - title; + - linked spec, if present; + - priority/dependencies, if visible; + - why it is safe/ready; + - verification surface from task body, if present; + - next command. +5. If no task exists, present a short “queue empty” card and recommend one of: + - `$coco-goal ` to create a real GoalSpec + backlog tasks; + - `$coco-night --plan-next` if the user wants coco to pick one safe task and then attempt it. + +## Output shape + +```text +◈ coco-queue · next ready task + Task + Title + Why <ready because...> + Proof <verification surface or missing> + Next $coco-loop + + source: coco_next · coco-store progress +``` + +Keep it concise. The user wants a queue decision, not a full plan. From 83a79edb334994b353ac43212731bc22b20fd7db Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:49:44 +0930 Subject: [PATCH 54/66] feat: add coco-night skill --- skills/coco-night/SKILL.md | 97 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 skills/coco-night/SKILL.md diff --git a/skills/coco-night/SKILL.md b/skills/coco-night/SKILL.md new file mode 100644 index 0000000..62e29ff --- /dev/null +++ b/skills/coco-night/SKILL.md @@ -0,0 +1,97 @@ +--- +name: coco-night +description: Use when the user types $coco-night or /coco-night, or says “work while I sleep”, “pick the next task and implement it”, “run coco overnight”, or “take one queued task”. Safe overnight workflow — pick exactly one ready backlog task, or optionally create one via goal planning, then run one bounded coco-loop. Stops at merge-gate unless the user explicitly opted into --auto. +--- + +# coco-night + +Run **one bounded overnight coco attempt**: choose exactly one ready task, implement it through `$coco-loop`, and stop safely. This is syntax sugar for: + +1. `$coco-queue next` / `coco_next` — pick one ready task; +2. `$coco-loop` — implement that task under coco’s normal Oracle-review + coco-owned verify + merge gate. + +It is not a daemon and not a scheduler. If the user later wants it on a timer, configure Codex app Automations to run this skill after the workflow is proven reliable. + +Invoke: + +```text +$coco-night +$coco-night --auto +$coco-night --plan-next +$coco-night <specific objective> +``` + +## What “night” means + +- **One task only.** No batch processing, no “keep going until morning.” +- **No silent merge.** Default stops at `merge-gate` for the human. `--auto` grants forward consent only for this one goal and still uses coco’s risk gate. +- **No broad exploration.** If the queue is empty, stop unless the user passed `--plan-next` or an explicit objective. +- **Wake-up summary.** End with what happened, where it stopped, and the exact next command for the human. + +## Golden rules + +1. **Preflight first.** Run `coco doctor` and `coco audit validate` when available. If doctor has a blocking failure, audit is invalid, verify config is missing, Oracle is missing, the tree is dirty, or a goal is already active, stop with recovery steps. +2. **One coherent work unit.** Pick one `ready` task from `coco_next`; if the user gave a concrete objective, use that instead. Never start a second task after the first finishes. +3. **Queue before invention.** If no objective is provided, call `coco_next` first. If no ready task exists, stop unless `--plan-next` was passed. +4. **Plan-next is bounded.** With `--plan-next`, create a small task using `$coco-goal "pick the next safest loop-sized task for this project"`, then implement the first promoted ready task only. Prefer docs/tests/refactors/diagnostics over risky feature work when the project has no explicit queue. +5. **Use `$coco-loop` exactly.** Once a task/objective is selected, follow the normal `$coco-loop` rules. Do not copy/paste or fork the loop protocol here. +6. **Do not bypass consent.** Default human path stops at merge-gate. `--auto` may call `coco_merge` only because the user explicitly granted per-goal consent and only if risk policy allows it. +7. **Respect stop conditions.** Stop on `review-unavailable`, `stuck`, `budget-exceeded`, repeated verify aborts, rebase conflicts, missing test command, or any human-choice ambiguity. +8. **Leave a crisp report.** The user should wake up to a useful summary, not a transcript dump. + +## Workflow + +1. **Parse flags.** + - `--auto`: pass `autoMergeAllowed:true` to goal start via `$coco-loop --auto`. + - `--plan-next`: allowed to create one next task if queue is empty. + - Any remaining text is a specific objective. +2. **Preflight.** + - `coco doctor` + - `coco audit validate` if the command exists + - `git status --porcelain` + - `coco health` if an active goal may exist +3. **Select work.** + - If objective text exists: use it as the loop objective. + - Else call `coco_next({ repoDir })` / `coco next`. + - If a task exists: use its title/body and remember its id. + - If none exists and `--plan-next` is false: stop with `$coco-goal` / `$coco-queue` suggestions. + - If none exists and `--plan-next` is true: run the `$coco-goal` workflow to archive/promote one small next task, then re-run `coco_next`. +4. **Run one loop.** + - Invoke/follow `$coco-loop` for exactly the selected task/objective. + - If a backlog task was selected, pass its id as `backlogTaskId` through the normal loop path. + - If `--auto` was passed, use the normal loop auto-merge path; do not create any new merge path here. +5. **Stop.** + - On `merge-gate`: present human merge command and stop. + - On auto-merge success: mark backlog task done through the normal loop path and stop. + - On blocker: summarize blocker and recovery command; stop. + +## Wake-up report + +Use this shape at the end: + +```text +◈ coco-night · one-task overnight run + Picked <task id/title or objective> + Result <merged | merge-gate | blocked | no-ready-task> + Proof <verify pass/fail or why not reached> + Stopped at <nextAction / blocker> + Next <exact human command or next skill> + + source: coco-night · one task only · no silent merge +``` + +## Recommended automation + +Only after `$coco-night` has worked manually, schedule it in Codex app Automations with a prompt like: + +```text +$coco-night +``` + +or, for opt-in low-risk auto-merge: + +```text +$coco-night --auto +``` + +Prefer a dedicated git worktree for scheduled runs so an overnight session cannot collide with your daytime working tree. From 99b397eb520909ea9da294ad16287cc9ee4a1d87 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:50:19 +0930 Subject: [PATCH 55/66] docs: register queue and night skills --- src/commands/registry.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/commands/registry.ts b/src/commands/registry.ts index 52a499a..0ff8faf 100644 --- a/src/commands/registry.ts +++ b/src/commands/registry.ts @@ -2,7 +2,7 @@ export type CommandEffect = 'read' | 'write' | 'destructive' | 'external'; export interface CommandDescriptor { name: string; - surfaces: ('cli' | 'mcp' | 'skill')[]; + surfaces: ('cli' | 'mcp' | 'skill')[ ]; effect: CommandEffect; summary: string; safety?: string; @@ -10,6 +10,9 @@ export interface CommandDescriptor { const COMMANDS: readonly CommandDescriptor[] = [ { name: 'coco init', surfaces: ['cli', 'mcp'], effect: 'write', summary: 'Bootstrap .coco and tracked coco.config.json on a clean repo.' }, + { name: 'coco next', surfaces: ['cli', 'mcp', 'skill'], effect: 'read', summary: 'Return the next ready backlog task according to dependency/priority rules.' }, + { name: '$coco-queue', surfaces: ['skill'], effect: 'read', summary: 'Inspect the project queue and explain the next ready task without implementing it.' }, + { name: '$coco-night', surfaces: ['skill'], effect: 'external', summary: 'Pick exactly one ready task and run one bounded coco-loop attempt for overnight work.', safety: 'Stops at merge-gate unless the user explicitly invoked --auto for this one goal.' }, { name: 'coco goal start', surfaces: ['cli', 'mcp', 'skill'], effect: 'write', summary: 'Create one active goal branch from the configured base branch.', safety: 'Refuses dirty trees and concurrent active goals.' }, { name: 'coco goal status', surfaces: ['cli', 'mcp', 'skill'], effect: 'read', summary: 'Derive deterministic nextAction from goal state and live git.' }, { name: 'coco goal record', surfaces: ['cli', 'mcp', 'skill'], effect: 'write', summary: 'Append plan/implement/review events bound to expectedSha.', safety: 'Review verdicts are parsed from Oracle output; verify is not accepted here.' }, From 9cfd1226654a23a0b53f8341d4da1766b32df779 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:50:58 +0930 Subject: [PATCH 56/66] fix: correct command descriptor surface type --- src/commands/registry.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/commands/registry.ts b/src/commands/registry.ts index 0ff8faf..05f4f07 100644 --- a/src/commands/registry.ts +++ b/src/commands/registry.ts @@ -2,7 +2,7 @@ export type CommandEffect = 'read' | 'write' | 'destructive' | 'external'; export interface CommandDescriptor { name: string; - surfaces: ('cli' | 'mcp' | 'skill')[ ]; + surfaces: ('cli' | 'mcp' | 'skill')[]; effect: CommandEffect; summary: string; safety?: string; From c85e7f4de8e03a0d471440219cc3c050071249d6 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:52:09 +0930 Subject: [PATCH 57/66] docs: rewrite README with modern workflow examples --- README.md | 289 +++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 220 insertions(+), 69 deletions(-) diff --git a/README.md b/README.md index 32942f5..82c3ec3 100644 --- a/README.md +++ b/README.md @@ -2,105 +2,253 @@ <img src="assets/coco-icon.svg" alt="coco" width="180" /> </p> -# coco +<h1 align="center">coco</h1> -**A minimal loop-engineering referee for AI coding agents** (Codex / Claude Code). +<p align="center"> + A tiny, strict referee for AI coding agents: plan → implement → review → verify → human merge. +</p> + +<p align="center"> + Codex / Claude Code do the work. Oracle reviews. coco refuses unsafe next steps. +</p> + +--- + +## What coco is + +coco is a local workflow layer for using coding agents every day without trusting chat memory. -coco is one small loop: **plan → implement → review → fix → verify → human-merge**, gated at every step by a git-**tree-hash**-bound, **epoch-scoped** state machine (an append-only event log; all gates derived; `coco merge` is the only merge path and is human-terminal by design). **ChatGPT-Pro ("Oracle")** is the review brain; the coding agent is the hands; **you** own the merge. The referee is deliberately tiny — it never writes your code, it just refuses the unsafe next step. +It gives you a small state machine around agentic development: -Three layers, each with a `$`/`/` skill trigger: +1. turn vague work into a verifiable goal; +2. put loop-sized tasks in a project queue; +3. let the agent implement exactly one task; +4. require strict Oracle review; +5. let coco run verification itself; +6. stop at a human merge gate, unless you explicitly opted into low-risk auto-merge for that one goal. -| Layer | Skill | Role | +coco is not a code generator. It is the guardrail around one. + +## Why it exists + +Coding agents are powerful, but long sessions fail in predictable ways: + +- the agent forgets what phase it is in; +- a review result from an old tree gets reused after new changes; +- tests are described as passing instead of actually run by the tool; +- merge consent becomes vague; +- background work drifts outside the intended scope; +- useful learnings disappear into chat history. + +coco makes those parts explicit and durable. The current HEAD/tree, review verdict, verify result, branch state, queue task, audit trail, and merge gate all live in local artifacts and deterministic commands. + +## The mental model + +| Layer | Skill / command | Job | |---|---|---| -| **CEO** | `$coco-goal` | turn a weak intent into a strong, achievable, archivable **GoalSpec** (Outcome · Verification surface · Constraints · Boundaries · Iteration · Blocked-stop) + loop-sized steps | -| **PM** | `$coco-store` / `coco-store` | organise / track / visualise the knowledge base — ResourceCards, roadmap, backlog progress, links, context `pack`, mermaid project graph | -| **CTO** | `$coco-loop` | drive the gated build loop with Oracle review + coco-owned verify | +| CEO | `$coco-goal` | Turn a fuzzy intent into a strong GoalSpec and loop-sized backlog tasks. | +| Queue | `$coco-queue` / `coco next` | Show the next ready task without implementing it. | +| CTO | `$coco-loop` | Implement one task through review, verify, and merge gate. | +| Night | `$coco-night` | Pick one ready task and run one bounded overnight attempt. | +| PM | `$coco-store` / `coco-store` | Specs, roadmap, backlog, links, context packs, project graph. | +| Improve | `$coco-improve` | Use valid audit + feedback to propose one eval-backed workflow improvement. | + +## Real-world examples + +### “I have a vague idea” -## coco is built with coco +```text +$coco-goal Make the project easier to use every day in Codex.app +``` -coco builds itself. Every feature below was driven through `$coco-loop` on coco's own repo — Oracle plans it, the agent implements, Oracle reviews the real diff, coco runs coco's own tests to verify, and a human merges. That dogfooding is the toughest test of the loop, and it works: Oracle's review gate has caught a genuine correctness bug in **every** phase — from a monotonic-verdict bypass to a `coco init` that scaffolded its config file *before* checking for staged changes (a working-tree leak), each found by reviewing the committed diff, not by trusting the author. The loop converges, the human still owns the merge, and coco never self-merges. +coco-goal reads the repo, researches current constraints when needed, writes a GoalSpec, archives it in `coco-store`, and promotes loop-sized tasks to the backlog. It does not start implementation. -As of **v0.4**, coco also **improves itself**: `coco-audit` records every meaningful loop action to a local trail, `coco-doctor` checks its own prerequisites and health, and `$coco-improve` reflects over that trail to draft ONE evidence-backed change — **archived, never applied**, for a human to run through the same plan → review → verify → merge gate. Crucially, the self-improvement loop is bound by the very referee it would improve: an improve-origin change can **never** merge an edit to coco's own gate, verdict parsing, verify, or metrics — enforced in code at both propose *and* merge time (via an `improveOrigin` flag frozen at goal start). coco can propose how to sharpen its own loop; it cannot quietly weaken the gate that keeps it honest. +### “What should I work on next?” -## What's new +```text +$coco-queue +``` -- **Daily Codex hardening** — root `AGENTS.md` captures durable repo/agent guidance, `.codex/config.toml.example` shows local MCP wiring, `coco setup codex` dry-runs/applies MCP + skill setup, and `docs/codex-daily-workflow.md` gives the Mac Codex.app daily path. -- **Referee state hardening** — persisted `.coco/goals/*.json` is schema-validated and stamped with a goal schema version instead of being blindly cast from JSON. -- **Configurable base branch + collision-safe goal ids** — new goals use `workflow.baseBranch` / repo default branch, with explicit `--base` override; repeated same-minute objectives get a safe numeric suffix instead of colliding with existing goal files/branches. -- **Verification-policy acknowledgement** — a goal that changes `verify.testCommand` can still be reviewed/verified, but human merge now requires explicit `--ack-verify-policy-change`; auto-merge falls back to that human path. -- **Additive auto-merge policy globs** — user `autoMerge.sensitiveGlobs` / `testGlobs` add to conservative defaults unless `replaceDefault*Globs` is explicitly set. -- **Command registry + deterministic evals** — `coco commands` exposes command metadata and `coco eval` runs lightweight deterministic safety-regression fixtures; `pnpm ci` includes the eval gate. -- **`coco-mcp` package bin** — the npm package exposes `coco-mcp` alongside `coco` and `coco-store`, matching the documented MCP setup path; the MCP server reports the package version. -- **CI-ready verification** — `pnpm typecheck`, `pnpm test`, `pnpm eval`, `pnpm build`, and `pnpm ci` are wired into a GitHub Actions matrix on Linux and macOS. -- **Privacy and platform docs** — `docs/privacy-model.md` documents what may leave the machine; `docs/platform-support.md` documents the current macOS/Linux support contract. -- **`coco-improve` — self-improving loop (propose-only)** — `$coco-improve` reflects over the `coco audit` corpus (with an **insufficient-data gate** so a thin history can't manufacture findings), forms ONE incremental, evidence-backed hypothesis, and archives it as a **local** improve-spec + one loop-sized backlog task for a human to run through `$coco-loop`. It never builds, merges, or edits live skills. -- **`coco-audit` — automatic trajectory capture** — every meaningful loop/goal action (reviews, fixes, verify results, Oracle outages, merges) is recorded to a local, gitignored `.coco/audit.ndjson` at the domain chokepoint. -- **`coco doctor` — one-shot health check** — a read-only, **no-LLM** diagnostic that aggregates environment (node/git/version), repo setup (init, `verify.testCommand`), wiring (merge-guard hooks, coco + Oracle MCP, watchdog), active-goal health, and data hygiene. `coco doctor clean` reclaims stale verify-run cache — **dry-run by default**, `--apply` to delete, and only terminal/orphaned runs. -- **Native `◈ coco` progress cards** — every layer surfaces a consistent, fenced **checkpoint card** so you can watch progress natively in the Codex app. -- **`coco-store` PM surface** — `list --group-by`, `progress` (backlog by status, grouped by spec), `link` (with a content-addressed links-merge), and `viz` (a structural mermaid project graph). -- **Context `pack`** wires the store → loop: the loop reads a bounded coco-store brief before it plans. +coco-queue reads the backlog and project store, shows the next ready task, explains why it is ready, and stops. -## Prerequisites +### “Implement the next task” -- **Node.js ≥ 20.** -- **macOS or Linux.** Windows native support is not guaranteed yet; use WSL. See `docs/platform-support.md`. -- **An MCP-aware coding agent** — [OpenAI Codex](https://developers.openai.com/codex) or [Claude Code](https://claude.ai/code) — with the `coco` (and `oracle`) MCP servers registered and the `coco-goal` / `coco-store` / `coco-loop` skills installed under the agent's skills dir. -- **A ChatGPT Pro subscription + Oracle — required for the review brain.** coco's plan/review gate runs **GPT‑5.x‑Pro** through the [`@steipete/oracle`](https://github.com/steipete/oracle) lane, which drives a logged-in **ChatGPT Pro** browser session (the `consult` MCP tool). Without an active **ChatGPT Pro** subscription and Oracle configured, the CLIs still run, but the Oracle-gated plan/review steps can't — by design coco then **fails to the human, never a false green**. +```text +$coco-loop +``` -> coco is opinionated and not turnkey: it assumes the wiring above. The `coco` / `coco-store` CLIs and the `coco-mcp` stdio server work standalone, but the full loop needs the agent + skills + Oracle in place. +coco-loop asks coco for the next ready backlog task and runs exactly one implementation loop. -## Install +### “Work on one thing while I sleep” -```sh -npm install -g @nickcao/coco +```text +$coco-night +``` + +coco-night is a safe wrapper around queue + loop. It picks one ready task, runs one bounded attempt, and leaves a wake-up report. By default it stops at the merge gate. + +For a low-risk task where you explicitly want one-goal auto-merge consent: + +```text +$coco-night --auto +``` + +Auto-merge still requires clean review, coco-owned verify, base ancestry, per-goal consent, and the risk policy. Risky work falls back to the human merge path. + +If the queue is empty and you want coco to pick one small next task: + +```text +$coco-night --plan-next ``` -Provides `coco` (the referee), `coco-store` (the PM layer), and `coco-mcp` (the MCP stdio server for MCP-aware agents). +That first creates one next task through the goal-planning path, then attempts only that task. -## Daily Codex.app setup +### “Improve coco using its own history” + +```text +$coco audit validate +coco audit feedback --goal <goalId> --kind goal-quality --rating 2 --tags vague-goal,weak-proof +$coco-improve +``` -For Codex.app, start from the durable guidance and examples in this repo: +coco-improve acts only on valid audit data and safe signals. It proposes one local improve-spec and one backlog task. It does not edit code or merge. + +## Install ```sh -cat AGENTS.md -cat .codex/config.toml.example -cat docs/codex-daily-workflow.md +npm install -g @nickcao/coco ``` -Then dry-run/apply setup and check health: +Requires Node.js 20 or newer and Git. + +For the full loop you also need: + +- OpenAI Codex or Claude Code; +- the `coco` MCP server configured; +- Oracle configured for deep review; +- the coco skills installed or synced; +- a committed `coco.config.json` with `verify.testCommand`. + +## First setup in a repo ```sh +cd your-repo +coco init coco setup codex coco setup codex --apply coco doctor ``` -The daily path is: +`coco init` creates `.coco/` runtime state and a starter `coco.config.json`. + +Set your verify command: + +```json +{ + "verify": { + "testCommand": "pnpm test", + "timeoutSec": 600, + "outputLimitBytes": 65536 + }, + "workflow": { + "baseBranch": "main" + } +} +``` + +## Daily workflow + +```sh +coco doctor # check local wiring +coco-store status # project pulse +coco next # next ready backlog task +``` + +Then use one of the skills: + +```text +$coco-goal <intent> # define and queue work +$coco-queue # inspect next task +$coco-loop # implement one queued task +$coco-night # one bounded overnight attempt +$coco-improve # propose one workflow improvement +``` + +## The safety contract -1. `$coco-goal <intent>` for vague or multi-step work. -2. `$coco-loop` for one ready backlog task, or `$coco-loop <objective>` for a specific task. -3. `coco-store status` / `coco-store progress` for PM state. -4. `$coco-improve` only to propose one evidence-backed improvement; it never edits or merges the proposal itself. +coco is intentionally conservative. -## Quick taste (CLI, no agent) +- Review verdicts must come from strict Oracle output parsing. +- Verify is coco-owned; the agent cannot self-report `pass`. +- Review and verify are bound to the current HEAD/tree. +- Dirty trees, wrong branches, stale reviews, missing verify config, and rebase needs block progress. +- Merge requires active goal, correct branch, clean tree, current-epoch implementation, clean review, passing verify, and base ancestry. +- Changing `verify.testCommand` requires explicit human acknowledgement with `--ack-verify-policy-change`. +- Auto-merge is opt-in per goal and still risk-gated. +- Self-improvement cannot touch protected referee/metrics/store/improve-self paths. +- Audit data is local, redacted, and validated before it drives improvement. + +## What the progress card looks like + +coco returns machine-readable status, and the skills echo compact cards in Codex.app / Claude Code: + +```text +◈ coco-loop · goal-20260708-2214-add-night-mode + Checkpoint ready — merge gate + Branch coco/goal-... → main (on goal) + Verified plan ✓ implement ✓ review ✓ clean verify ✓ pass + Remaining merge + Risk — + Recovery coco merge --goal goal-20260708-2214-add-night-mode + Next merge-gate — awaiting human approval + + source: goal-... · a1b2c3d · state=active · next=merge-gate +``` + +The merge command is still a separate human approval step. + +## CLI cheat sheet ```sh -cd your-repo -coco init # bootstrap .coco/ + a starter coco.config.json on a clean main -coco setup codex # dry-run local Codex MCP + skill setup -coco commands # command registry / effects overview -coco eval # deterministic safety-regression fixtures -# ... the loop is normally driven by the $coco-loop skill, not by hand ... -coco-store progress # backlog by status, grouped by spec -coco-store status # native ◈ project-pulse card (backlog · specs · roadmap) -coco-store viz # mermaid project graph -coco audit report # loop trajectory: fix-rounds, verify fails, oracle outages, merge latency -coco doctor # health + prereqs (`coco doctor clean` reclaims stale verify-run cache) -coco improve digest # audit-derived pain signals (insufficient-data-gated) -coco improve check <paths> # refuse edits to the referee/metrics/store (exit 3) +# repo setup +coco init +coco setup codex +coco doctor + +# queue and loop +coco next +coco goal status --goal <goalId> +coco goal verify --goal <goalId> --expected-sha <sha> +coco merge --goal <goalId> +coco merge --goal <goalId> --ack-verify-policy-change + +# audit and self-improvement +coco audit report +coco audit validate +coco audit feedback --goal <goalId> --kind status-clarity --rating 5 --tags clear-card +coco improve digest +coco improve check <paths> +coco eval + +# PM layer +coco-store status +coco-store progress +coco-store viz +coco-store pack --goal <goalId> --query "<objective>" ``` -## Development verification +## Files and directories + +| Path | Purpose | +|---|---| +| `.coco/` | Local runtime state, goal ledgers, verify runs, audit logs. Gitignored. | +| `.coco-store/` | Local PM store: specs, resources, links, roadmap context. | +| `coco.config.json` | Committed repo policy: verify command, workflow base branch, auto-merge policy. | +| `AGENTS.md` | Durable repo guidance for Codex/agents. | +| `skills/` | coco skills to install/sync into your agent skills directory. | +| `docs/self-evolution.md` | How audit + feedback become eval-backed improvement proposals. | + +## Development ```sh pnpm install @@ -111,17 +259,20 @@ pnpm build pnpm ci ``` -`pnpm ci` is the pre-PR gate: typecheck, test, deterministic evals, then build. GitHub Actions runs the same gate on Linux and macOS. +GitHub Actions runs the CI gate on Ubuntu and macOS with Node 22/24. + +## Platform support + +coco is macOS/Linux-first. Windows native support is not guaranteed yet; use WSL. See `docs/platform-support.md`. -## Privacy and platform +## Privacy -- Read `docs/privacy-model.md` before adding new context/Oracle surfaces. -- Read `docs/platform-support.md` before claiming support beyond macOS/Linux. +The full loop can send bounded context and diffs to Oracle for plan/review. Local audit logs, local store cards, runtime state, and secrets must not be sent. See `docs/privacy-model.md`. ## Credits -- **Review + plan brain — [Oracle (`@steipete/oracle`)](https://github.com/steipete/oracle)** by [Peter Steinberger](https://github.com/steipete): the lane that drives **ChatGPT Pro (GPT‑5.x‑Pro)** for deep review, planning, and research. coco's "never false-green" gate is only as strong as this brain — coco leans on it at every plan and review step, and wouldn't exist without it. (Requires your own ChatGPT Pro subscription.) -- **Icon** generated with ChatGPT, via Oracle. +- Review + plan brain: Oracle (`@steipete/oracle`) with ChatGPT Pro. +- Icon generated with ChatGPT and refined for the comic-style coco mascot. ## License From cb8d9fac059c8679c60528d871fe31aa2280e99b Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:53:03 +0930 Subject: [PATCH 58/66] docs: add queue and night workflows --- docs/codex-daily-workflow.md | 40 ++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/docs/codex-daily-workflow.md b/docs/codex-daily-workflow.md index 0662ee4..ad68f62 100644 --- a/docs/codex-daily-workflow.md +++ b/docs/codex-daily-workflow.md @@ -44,9 +44,10 @@ Run: ```sh coco doctor coco-store status +coco next ``` -Use the doctor output to catch broken local wiring before starting a long loop. Use the store status card to see backlog/spec state without reading the whole repository. +Use the doctor output to catch broken local wiring before starting a long loop. Use the store status card and next-task command to see backlog/spec state without reading the whole repository. ## Turning intent into work @@ -58,6 +59,16 @@ $coco-goal <intent> The goal skill should read the repo, research current external constraints where relevant, produce a strong GoalSpec, archive it as a `coco-store` spec, promote loop-sized tasks to `BACKLOG.md`, and stop. +## Choosing from the queue + +To inspect the next ready task without implementing it: + +```text +$coco-queue +``` + +This is read-only. It calls the code-owned queue first (`coco next` / `coco_next`), summarizes why the task is ready, and stops with the next suggested command. + ## Building one task For one implementation task, use: @@ -80,6 +91,30 @@ $coco-loop --auto <objective> Auto-merge is still gated by clean review, coco-owned verify, base ancestry, risk policy, unchanged verification policy, and per-goal consent. Risk or verify-policy fallback returns to the human merge path. +## Overnight one-task mode + +For a bounded “work while I sleep” attempt: + +```text +$coco-night +``` + +`$coco-night` is queue + loop, not a daemon. It picks exactly one ready task, runs one normal coco-loop attempt, and leaves a wake-up report. By default it stops at `merge-gate`. + +Only use auto when you explicitly want per-goal forward consent: + +```text +$coco-night --auto +``` + +If the queue is empty and you want coco to create one small next task first: + +```text +$coco-night --plan-next +``` + +Use Codex app Automations only after `$coco-night` works manually; skills define the method, automations define the schedule. + ## Base branch New goals branch from `workflow.baseBranch` in `coco.config.json` when set. Otherwise coco resolves the repo default branch (`origin/HEAD`, then `main`/`master`/`trunk`/`develop`, then the current branch). Use an explicit base only for unusual work: @@ -112,7 +147,7 @@ Run: $coco-improve ``` -only when there is enough audit history. The improve skill should propose exactly one local improve-spec and one non-protected backlog task, then stop. It must not edit live code, start a loop, or merge. +only when there is enough valid audit history. The improve skill should propose exactly one local improve-spec and one non-protected backlog task, then stop. It must not edit live code, start a loop, or merge. ## Pre-PR verification @@ -133,6 +168,7 @@ pnpm ci - `coco doctor` reports `oracle MCP` missing: configure Oracle before plan/review gates. - `goal status` warns about `verify.testCommand`: set `verify.testCommand` in committed `coco.config.json`. +- `$coco-night` finds no ready task: run `$coco-goal <intent>` or `$coco-night --plan-next`. - `merge` refuses with `ack-verify-policy-change`: verify policy changed in the goal diff; inspect it, then approve the explicit acknowledgement command only if intended. - `health` reports `review-unavailable`: fix Oracle login/wiring, then resume with `coco_goal_op_clear`. - `health` reports `stalled`: inspect the current `nextAction`; do not guess the phase from chat memory. From 409a53b7ae834d6e76c1a6288d70c4ba947c0b60 Mon Sep 17 00:00:00 2001 From: xic <134991273+xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 18:54:00 +0930 Subject: [PATCH 59/66] docs: add queue and night workflow guidance --- AGENTS.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 86e50c5..b573688 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,7 @@ This repository builds `coco`, a small loop-engineering referee for AI coding ag - `src/store/**` is the PM/knowledge layer. It must not import or mutate goal/referee state. - `src/improve/**` implements propose-only self-improvement guards and audit-derived signals. - `src/commands/eval.ts` contains deterministic safety-regression fixtures. Add a fixture when fixing a state-machine or false-green class. -- `skills/**` contains the user-facing Codex/Claude skills. +- `skills/**` contains the user-facing Codex/Claude skills, including queue/night orchestration. Skill text is part of the safety surface. - `.coco/**` is local runtime state and must not be committed. - `.coco-store/**` is the local PM store. Treat audit-derived/self-improvement content as local unless code explicitly marks it shared. @@ -33,6 +33,7 @@ Useful local product commands: ```sh coco commands +coco next # next ready backlog task coco setup codex # dry-run Codex MCP + skill setup coco setup codex --apply # apply local setup coco eval # deterministic safety fixtures @@ -52,6 +53,7 @@ Do not weaken these without an explicit, human-authored referee-change goal and 8. Auto-merge is opt-in per goal and must never loosen the human merge path. 9. Improve-origin changes must not touch protected referee, metrics, store, or improve-self paths. 10. Store/pack must preserve the privacy boundary: local cards and secret-looking files must not be sent to Oracle. +11. `$coco-night` may run at most one task and must stop at merge-gate unless the user explicitly supplied `--auto` for that one run. ## Coding style @@ -75,6 +77,8 @@ For normal development in Codex.app: 1. Run `coco setup codex` once as a dry-run, then `coco setup codex --apply` if the paths are correct. 2. Run `coco doctor` before a long session. 3. Use `$coco-goal` for vague or multi-step work. -4. Use `$coco-loop` for one loop-sized implementation task. -5. Use `$coco-store status` or `coco-store progress` for project state. -6. Use `$coco-improve` only to propose one evidence-backed improvement; it must not build, edit, or merge the proposal itself. +4. Use `$coco-queue` or `coco next` to inspect the next ready task without implementing. +5. Use `$coco-loop` for one loop-sized implementation task. +6. Use `$coco-night` for one bounded overnight attempt; schedule it as an automation only after it works manually. +7. Use `$coco-store status` or `coco-store progress` for project state. +8. Use `$coco-improve` only to propose one evidence-backed improvement; it must not build, edit, or merge the proposal itself. From 1c3bb0e54dd8e95144a6974d1d8509cad2d47a65 Mon Sep 17 00:00:00 2001 From: nickcao <xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 19:40:16 +0930 Subject: [PATCH 60/66] fix(schema): accept empty failureLoop.key sentinel on reload MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The fingerprint tracker returns key:'' (count:0) as the "no active failure" sentinel after a resolve or an initial non-failure event. The new goalSchema required key to be non-empty, so every goal persisted with a reset failure-loop failed to reload — a false-block that broke the plan→implement→review→verify path (58 tests). Allow an empty top-level key; history entries stay non-empty since only real failure signatures are ever appended there. Claude-Session: https://claude.ai/code/session_01UQ775a5fjQWYUF522M7snK --- src/goalSchema.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/goalSchema.ts b/src/goalSchema.ts index f5e330a..f52467f 100644 --- a/src/goalSchema.ts +++ b/src/goalSchema.ts @@ -42,7 +42,10 @@ const reviewUnavailableSchema = z const failureLoopSchema = z .object({ - key: z.string().min(1), + // Empty string is the intentional sentinel for "no active failure track" (fingerprint.ts + // returns key:'' with count:0 on reset/resolve). History entries below are only ever appended + // for real failures, so those keep the non-empty constraint. + key: z.string(), count: z.number().int().nonnegative(), history: z.array( z From 73e5534943eea884be0eefc78b44d9ddc7e51273 Mon Sep 17 00:00:00 2001 From: nickcao <xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 19:40:22 +0930 Subject: [PATCH 61/66] fix(eval): repair fingerprint-loop-escalates fixture, add reload guard Two eval fixes surfaced once CI actually ran the suite: - fingerprint-loop-escalates asserted escalate-human but supplied no plan event, so nextAction short-circuited to 'plan' and never reached the fingerprint branch. Add the missing plan precondition. - Add reset-failure-loop-survives-reload: rounds a goal with the empty-key sentinel through parseGoalState so the schema false-block cannot regress. Claude-Session: https://claude.ai/code/session_01UQ775a5fjQWYUF522M7snK --- src/commands/eval.ts | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/src/commands/eval.ts b/src/commands/eval.ts index ac8705d..ab784eb 100644 --- a/src/commands/eval.ts +++ b/src/commands/eval.ts @@ -3,8 +3,9 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { appendAudit, auditPath } from '../audit.js'; import { deriveFacts } from '../epoch.js'; -import { FINGERPRINT_N } from '../fingerprint.js'; +import { FINGERPRINT_N, updateFailureLoop } from '../fingerprint.js'; import { nextAction } from '../gate.js'; +import { parseGoalState } from '../goalSchema.js'; import { parseOracleVerdict } from '../oracleVerdict.js'; import type { GoalEvent, GoalState } from '../types.js'; import { auditValidate } from './audit.js'; @@ -90,7 +91,9 @@ const cases: EvalCase[] = [ run: () => nextAction( activeGoal({ - events: [ev('implement', 't1'), ev('review', 't1', 'blocking')], + // plan → implement is required for nextAction to reach the review/fingerprint branch; + // without a plan event it would short-circuit to 'plan' and never test escalation. + events: [ev('plan', 't1'), ev('implement', 't1'), ev('review', 't1', 'blocking')], failureLoop: { key: 'same-failure', count: FINGERPRINT_N, history: [] }, }), { tHead: 't1', treeClean: true, onBranch: true, baseMerged: true }, @@ -114,6 +117,18 @@ const cases: EvalCase[] = [ invariant: 'Invalid/torn audit lines and impossible merge ordering are detected before self-improvement trusts them.', run: auditInvalidFixture, }, + { + id: 'reset-failure-loop-survives-reload', + area: 'gate', + invariant: 'A goal whose failure-loop reset to the empty-key sentinel must survive persist→reload; the schema must not false-block valid runtime state.', + run: () => { + // updateFailureLoop returns key:'' after a non-failure event (the "no active failure" sentinel). + const reset = updateFailureLoop(undefined, ev('implement', 't1')); + if (reset.key !== '' || reset.count !== 0) return false; + const roundTripped = parseGoalState(JSON.parse(JSON.stringify(activeGoal({ failureLoop: reset })))); + return roundTripped.failureLoop?.key === '' && roundTripped.failureLoop.count === 0; + }, + }, ]; export function runEval(): EvalReport { From d99a83690e29b0ccb33fd02bb616f7bf1eb52d1d Mon Sep 17 00:00:00 2001 From: nickcao <xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 19:40:27 +0930 Subject: [PATCH 62/66] test(improve): audit-validity signal is intentionally not sample-gated MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The thin-history test asserted every signal is insufficient-data, but the added audit-validity signal deliberately fires regardless of window size (a corrupt audit substrate must halt self-improvement). Assert the real invariant — no signal is actionable on a thin history — and exempt audit-validity from the sample-gate check while keeping it non-actionable. Claude-Session: https://claude.ai/code/session_01UQ775a5fjQWYUF522M7snK --- tests/improve.test.ts | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/improve.test.ts b/tests/improve.test.ts index 40bdeb9..fd195d6 100644 --- a/tests/improve.test.ts +++ b/tests/improve.test.ts @@ -95,7 +95,12 @@ test('improveDigest gates on sample size: too few goals → insufficient-data, w seedGoals(repo, 2); const d = improveDigest(repo); expect(d.sufficient).toBe(false); - expect(d.signals.every((s) => s.status === 'insufficient-data' && s.safeToActOn === false)).toBe(true); + // The core guarantee: a thin history can never surface an ACTIONABLE finding. + expect(d.signals.every((s) => s.safeToActOn === false)).toBe(true); + // Every sample-gated signal reports insufficient-data. audit-validity is deliberately exempt — + // a corrupt audit substrate must halt self-improvement regardless of window size — but it is + // still non-actionable (safeToActOn === false, asserted above). + expect(d.signals.filter((s) => s.key !== 'audit-validity').every((s) => s.status === 'insufficient-data')).toBe(true); expect(d.antiGoals.length).toBeGreaterThan(0); }); From 1a0495b19befa8074d0985393726c9d3d143221f Mon Sep 17 00:00:00 2001 From: nickcao <xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 19:40:34 +0930 Subject: [PATCH 63/66] fix(ci): run `pnpm run ci`, not the built-in `pnpm ci` pnpm >=11 ships a built-in `ci` command (clean-install) that shadows the package script, so `pnpm ci` silently reinstalls instead of running typecheck/test/eval/build. The GitHub Actions gate was therefore green without ever verifying anything, masking real test and eval failures. Switch the workflow and all docs (AGENTS.md, README, codex-daily-workflow) to `pnpm run ci`, and fix a `$coco audit validate` README typo (CLI command, no skill prefix). Claude-Session: https://claude.ai/code/session_01UQ775a5fjQWYUF522M7snK --- .github/workflows/ci.yml | 4 +++- AGENTS.md | 7 +++++-- README.md | 4 ++-- docs/codex-daily-workflow.md | 7 +++++-- 4 files changed, 15 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0fdaf56..adb2d5b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -38,4 +38,6 @@ jobs: run: pnpm install --frozen-lockfile - name: Verify - run: pnpm ci + # `pnpm run ci`, NOT `pnpm ci`: pnpm >=11 ships a built-in `ci` command (clean-install) + # that shadows the package script, so `pnpm ci` would silently skip typecheck/test/eval/build. + run: pnpm run ci diff --git a/AGENTS.md b/AGENTS.md index b573688..71cb882 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,10 +24,13 @@ pnpm typecheck pnpm test pnpm eval pnpm build -pnpm ci +pnpm run ci ``` -`pnpm ci` is the expected pre-PR verification command. If a change touches a safety-critical gate, run the most specific affected tests as well as `pnpm ci`. +`pnpm run ci` is the expected pre-PR verification command (typecheck + test + eval + build). Use +`pnpm run ci`, not `pnpm ci`: pnpm >=11 ships a built-in `ci` command (clean-install) that shadows +the package script, so `pnpm ci` silently reinstalls instead of verifying. If a change touches a +safety-critical gate, run the most specific affected tests as well as `pnpm run ci`. Useful local product commands: diff --git a/README.md b/README.md index 82c3ec3..537fa64 100644 --- a/README.md +++ b/README.md @@ -106,7 +106,7 @@ That first creates one next task through the goal-planning path, then attempts o ### “Improve coco using its own history” ```text -$coco audit validate +coco audit validate coco audit feedback --goal <goalId> --kind goal-quality --rating 2 --tags vague-goal,weak-proof $coco-improve ``` @@ -256,7 +256,7 @@ pnpm typecheck pnpm test pnpm eval pnpm build -pnpm ci +pnpm run ci # typecheck + test + eval + build (use `run` — pnpm's built-in `ci` reinstalls instead) ``` GitHub Actions runs the CI gate on Ubuntu and macOS with Node 22/24. diff --git a/docs/codex-daily-workflow.md b/docs/codex-daily-workflow.md index ad68f62..e7c5527 100644 --- a/docs/codex-daily-workflow.md +++ b/docs/codex-daily-workflow.md @@ -154,10 +154,13 @@ only when there is enough valid audit history. The improve skill should propose Before a branch is ready for review, run: ```sh -pnpm ci +pnpm run ci ``` -`pnpm ci` runs typecheck, tests, the deterministic `coco eval` safety fixtures, and build. For referee-critical changes, also run targeted tests for the exact invariant being changed or protected. +`pnpm run ci` runs typecheck, tests, the deterministic `coco eval` safety fixtures, and build. Use +`pnpm run ci`, not `pnpm ci` — pnpm >=11 ships a built-in `ci` (clean-install) command that shadows +the package script and would silently reinstall instead of verifying. For referee-critical changes, +also run targeted tests for the exact invariant being changed or protected. ## Privacy and platform notes From 3959c8f0d9700db39176fc233ad48b90c2764ddc Mon Sep 17 00:00:00 2001 From: nickcao <xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 21:12:51 +0930 Subject: [PATCH 64/66] ci: bump actions to @v5 and fix prepublishOnly verification MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - actions/checkout and actions/setup-node → @v5 (they target Node 24 natively, clearing the Node-20-deprecation annotation forced onto the v4 actions). - prepublishOnly: `pnpm run ci`, not `pnpm ci` — the built-in `pnpm ci` (clean-install) shadows the script, so publish would otherwise skip typecheck/test/eval/build. Claude-Session: https://claude.ai/code/session_01UQ775a5fjQWYUF522M7snK --- .github/workflows/ci.yml | 4 ++-- package.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index adb2d5b..5f2867e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -23,13 +23,13 @@ jobs: steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@v5 - name: Enable Corepack run: corepack enable - name: Setup Node - uses: actions/setup-node@v4 + uses: actions/setup-node@v5 with: node-version: ${{ matrix.node }} cache: pnpm diff --git a/package.json b/package.json index 1ee4c45..aca32db 100644 --- a/package.json +++ b/package.json @@ -42,7 +42,7 @@ "eval": "tsx src/bin/coco.ts eval", "ci": "pnpm typecheck && pnpm test && pnpm eval && pnpm build", "dev": "tsx src/bin/coco.ts", - "prepublishOnly": "pnpm ci" + "prepublishOnly": "pnpm run ci" }, "devDependencies": { "@types/node": "^22.0.0", From 780db4aca36ad86329ffc915917d90e9c6561be8 Mon Sep 17 00:00:00 2001 From: nickcao <xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 21:14:54 +0930 Subject: [PATCH 65/66] =?UTF-8?q?chore(release):=20v0.6.0=20=E2=80=94=20Co?= =?UTF-8?q?dex=20daily=20productization,=20night/queue,=20self-evolution?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds $coco-night (one-task overnight), $coco-queue (read-only next-task triage), coco setup codex, audit validate/feedback, improve digest, configurable base branch, richer progress cards, and a modern README — plus the false-green CI fix that makes the gate actually verify. Claude-Session: https://claude.ai/code/session_01UQ775a5fjQWYUF522M7snK --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index aca32db..e11a875 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@nickcao/coco", - "version": "0.5.0", + "version": "0.6.0", "description": "A loop-engineering referee for AI coding agents (Codex / Claude Code): plan → implement → review → verify → human-merge, gated by a git-tree-hash-bound, epoch-scoped state machine, with ChatGPT-Pro (Oracle) as the review brain.", "type": "module", "license": "MIT", From e4009a3512e352d568277b8ce6c9c1d1d00e640e Mon Sep 17 00:00:00 2001 From: nickcao <xicv@users.noreply.github.com> Date: Wed, 8 Jul 2026 21:22:49 +0930 Subject: [PATCH 66/66] test(e2e): give the strict-parser e2e test the same 30s timeout The strict-Oracle-parser e2e test spawns ~6 tsx subprocesses but relied on vitest's default 5s timeout, so it flaked on the loaded macOS Node 24 runner (the sibling happy-path test already uses 30s). Align them. Claude-Session: https://claude.ai/code/session_01UQ775a5fjQWYUF522M7snK --- tests/e2e.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/e2e.test.ts b/tests/e2e.test.ts index 6886127..be56f13 100644 --- a/tests/e2e.test.ts +++ b/tests/e2e.test.ts @@ -53,7 +53,7 @@ test('end-to-end happy path via the CLI reaches merge-gate then merges', { timeo expect(JSON.parse(coco(repo, ['merge', '--goal', id])).merged).toBe(true); }); -test('CLI review goes through the strict Oracle parser — no --verdict false-green backdoor', () => { +test('CLI review goes through the strict Oracle parser — no --verdict false-green backdoor', { timeout: 30000 }, () => { const repo = mkdtempSync(join(tmpdir(), 'coco-e2e-')); coco(repo, ['init']); const id = JSON.parse(coco(repo, ['goal', 'start', '--objective', 'guard'])).goalId;