diff --git a/CHANGELOG.md b/CHANGELOG.md index 986dcb0e..6a6e2dd7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,18 +6,33 @@ All notable changes to vouch are documented here. Format follows ## [Unreleased] -## [1.2.2] — 2026-07-07 +### Added +- `kb.triage_pending` — advisory triage scoring over the pending-review queue. + Scores each pending proposal on fit, citation quality, duplication risk, and + contradiction risk, then attaches a `_meta.vouch_triage` block + (`recommendation`, `score`, `signals`, `rationale`) to help a reviewer + prioritize a long `kb.list_pending`. Read-only and advisory only: it never + calls `kb.approve` / `kb.reject` and never moves a proposal out of pending — + a human still decides. Duplication and fit reuse the propose-time embedding + similarity path and degrade to a `difflib` heuristic when the `[embeddings]` + extra isn't installed. Opt-in via `triage.enabled: true` in `config.yaml`. + `vouch triage [proposal-id...]` mirrors it on the CLI with `--json` and + `--reverse` (#322). +- mcp now serves a **minimal tool profile by default** (8 core tools) + instead of the full 58-method surface; widen with + `VOUCH_TOOL_PROFILE=standard|full` or `mcp.tool_profile` in + `config.yaml`. approve/reject and maintenance tools live in + `standard`/`full` — they stay human/cli actions, not agent defaults. + the jsonl and cli surfaces are unaffected; all 58 methods remain + reachable there. +- per-prompt auto-recall: the claude-code adapter's `UserPromptSubmit` + hook (`vouch context-hook`) injects relevant kb context on every + prompt, so recall no longer depends on the agent remembering to ask. -### Packaging -- published to the mcp registry (`registry.modelcontextprotocol.io`, mirrored - at `github.com/mcp/vouchdev/vouch`) as `io.github.vouchdev/vouch`. a - `server.json` at the repo root carries the metadata; the pypi `vouch-kb` - package is the artifact, run over stdio via `uvx vouch-kb serve`. -- `vouch-kb` console-script alias (alongside `vouch`) so `uvx vouch-kb serve` - resolves — the registry launches a package by its pypi identifier, which - otherwise wouldn't match the `vouch` script name. -- README carries an `` marker; - the registry verifies package ownership by matching it against `server.json`. +### Changed +- retrieval `auto`/`hybrid` now **fuses embedding + fts5** results via + reciprocal rank fusion instead of a waterfall (embedding-first, + fts5-fallback), with near-duplicate suppression over the fused list. ## [1.2.1] — 2026-07-06 @@ -111,6 +126,14 @@ All notable changes to vouch are documented here. Format follows ## [1.1.0] — 2026-07-03 ### Added +- `kb.diff` — `vouch diff ` (0.1.0) now has full `kb.*` + parity: an MCP tool and a JSONL `kb.diff` handler alongside the existing + CLI, registered in `capabilities.METHODS` like every other read method. + `` is now optional for a claim: omitting it resolves the diff + against `superseded_by`, with a clear error when the claim has no + successor (pages still require an explicit `new_id` — they have no + successor pointer). Read-only throughout; still no writes, proposals, or + audit events (#327). - auto-capture: claude code sessions are harvested via hooks and filed as a single pending session-summary proposal for human approval. a `PostToolUse` hook (`vouch capture observe`) appends compact tool-use observations to an @@ -212,6 +235,12 @@ All notable changes to vouch are documented here. Format follows temporary copied home containing known Claude/Codex credential files, so agent writes stay in the throwaway dual-solve branches and host credential files are not modified. +- dual-solve JSON, review-ui job, and choose responses now include + `changed_files` for each candidate and the kept branch, so desktop and browser + clients can show the resulting files without parsing unified diffs. +- dual-solve JSON and review-ui job responses now include each engine's returned + output log plus a deterministic recommendation hint based on success and diff + scope, so clients can compare Claude and Codex results before choosing. - `vouch review-ui --allow-dual-solve` — a browser SPA that runs `dual-solve` on a github issue link, streams progress over the review-ui's websocket, shows both engines' diffs side by side, and lets you pick the winner. Off by default; @@ -234,6 +263,7 @@ All notable changes to vouch are documented here. Format follows New `vouch schema list` and `vouch schema sync` commands inspect declared kinds and audit existing pages against them; `propose-page` gains `--kind` and repeatable `--meta key=value`. +- `kb.capabilities` now reports a `host_compat` block mirroring the `openclaw.compat` constraints declared in `openclaw.plugin.json` (e.g. `pluginApi`), so non-OpenClaw clients can detect compat without parsing the manifest. A new test asserts the two stay in sync and fails CI on drift (#237). - `kb.synthesize` — answer-mode retrieval over the review-gated KB. Answers a query in prose from approved claims only, with an inline `[claim_id]` citation behind every sentence, an explicit `gaps` block listing query @@ -266,6 +296,7 @@ All notable changes to vouch are documented here. Format follows KB under `eval/fixture-kb/`, and an `eval` workflow gating retrieval changes (#226). ### Fixed +- `Claim.text`, `Entity.name`, and `Page.title` now reject empty / whitespace-only values at the model layer via `@field_validator`s, mirroring the `Claim.evidence` min-citation validator (#81/#82). The non-empty contract previously lived only in the `propose_*` helpers, so `store.put_*`, `store.update_*`, and bundle/sync import all silently accepted blank-content artifacts; enforcing on the model closes every write path at once (fixes #155). - `parse_since` (the `--since` parser behind `vouch metrics`/`vouch audit`) now raises a clean `MetricsError` for a duration too large to represent (e.g. `--since 1000000000000d`), instead of letting an uncaught `OverflowError` traceback escape — restoring the documented "clean error, not a traceback" contract. - `sync_apply` now loads the sync source exactly once and passes the same `_SyncSource` instance into `sync_check`, closing a TOCTOU window where a bundle replaced on disk between the two `_load_source` calls could cause the validation and write phases to operate on different snapshots. Also eliminates redundant directory walks (KB sources) and triple tarball opens (bundle sources). Fixes #217. - `vault_to_kb` now passes `slug_hint=page_id` to `propose_page` so vault edit proposals target the existing page id from frontmatter instead of a slugified copy of the title (fixes #219). @@ -273,6 +304,8 @@ All notable changes to vouch are documented here. Format follows - `vault_to_kb` skips filing a second proposal when a pending proposal already targets the same page id (with differing body), preventing duplicate proposals on repeated sync runs before approval (fixes #219). - `vault_to_kb` now warns when a user edits a claim stub instead of silently dropping the edit, directing the user to edit the citing page instead, and reports it via the dedicated `claim_stubs_edited` field on `VaultSyncResult` (fixes #219). - `approve()` now supports updating an existing page via `KBStore.update_page` when a PAGE proposal's id matches an existing artifact (the vault-edit flow), instead of raising `cannot approve: page already exists` for every vault edit (fixes #219). +- `crystallize()` now updates an existing session summary page on retry instead of + failing with `page session-... already exists` after a partial approval run (#139). ### Fixed - `vouch serve` now fails fast with a clear `vouch init` hint when no `.vouch/` KB is present, instead of starting a server that immediately misbehaves (#95). diff --git a/README.md b/README.md index d076aa7e..bdb6a0ff 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,7 @@ vouch init vouch install-mcp claude-code ``` -`init` creates `.vouch/` with a starter config; `install-mcp` writes `.mcp.json` (the `kb.*` MCP tools), the `/vouch-*` slash commands, and three hooks — `PostToolUse` capture, `SessionEnd` rollup, `SessionStart` recall. Restart Claude Code so they load. +`init` creates `.vouch/` with a starter config; `install-mcp` writes `.mcp.json` (the `kb.*` MCP tools — a lean default surface you can widen with `VOUCH_TOOL_PROFILE`), the `/vouch-*` slash commands, and four hooks — `PostToolUse` capture, `SessionEnd` rollup, `SessionStart` recall, and `UserPromptSubmit` per-prompt context injection. Restart Claude Code so they load. **2. Point `compile` at an LLM** — the only step that needs a model. In `.vouch/config.yaml`: @@ -125,6 +125,8 @@ Detection is Claude Code's hook contract: whatever a `SessionStart` hook prints Only approved artifacts are ever emitted — archived, superseded, and still-pending items are excluded — and the digest is size-guarded (`recall.max_chars`) with an explicit truncation notice. +Recall isn't only a session-start event anymore: a `UserPromptSubmit` hook runs `vouch context-hook` on *every* prompt, fusing the most relevant approved claims and pages for what you just asked and injecting them with zero tool calls — so the model sees relevant knowledge even when the agent doesn't think to look it up. + How the approved pages actually get used from there: recall carries the *titles*, and the session pulls full content on demand through the `kb.*` MCP tools — `kb_search` matches page bodies, `kb_read_page` returns a page's markdown plus the claims it cites, and `kb_context` bundles the most relevant claims and pages for a stated task. To pull a topic in explicitly, use the `/vouch-recall ` slash command, or just ask Claude to check the KB. One thing to know: pages still sitting in `vouch review` are invisible to all of this — the gate applies to retrieval too, so a compiled page only starts informing sessions once you approve it. **7. Commit the knowledge with the code.** diff --git a/adapters/README.md b/adapters/README.md index f4edce8c..84bb8127 100644 --- a/adapters/README.md +++ b/adapters/README.md @@ -10,7 +10,7 @@ file you need into your project and edit it. |---|---|---| | [claude-code/](claude-code/) | Anthropic's Claude Code CLI | `.mcp.json` snippet, `CLAUDE.md` excerpt | | [cursor/](cursor/) | Cursor IDE | `mcp.json` snippet | -| [codex/](codex/) | OpenAI's Codex CLI | `config.toml` snippet | +| [codex/](codex/) | OpenAI's Codex CLI | tiered install: `.codex/config.toml` merge, `AGENTS.md` excerpt, skills, capture hook | | [continue/](continue/) | Continue.dev | `config.json` snippet | | [openclaw/](openclaw/) | OpenClaw plugin host | `.openclaw/plugins.json`, `AGENTS.md` excerpt | | [generic-mcp/](generic-mcp/) | Any MCP-speaking host | annotated reference | diff --git a/adapters/claude-code/.claude/settings.json b/adapters/claude-code/.claude/settings.json index 5fd35a67..d77ad5ad 100644 --- a/adapters/claude-code/.claude/settings.json +++ b/adapters/claude-code/.claude/settings.json @@ -43,6 +43,18 @@ ] } ], + "UserPromptSubmit": [ + { + "comment": "inject KB context relevant to THIS prompt (per-prompt auto-recall, 0 tool calls); never blocks the turn", + "matcher": "*", + "hooks": [ + { + "type": "command", + "command": "vouch context-hook || true" + } + ] + } + ], "PostToolUse": [ { "matcher": "*", diff --git a/adapters/claude-code/README.md b/adapters/claude-code/README.md index 8c3efe05..8739da73 100644 --- a/adapters/claude-code/README.md +++ b/adapters/claude-code/README.md @@ -112,3 +112,6 @@ To upgrade or check your extension version, see [Claude Code releases](https://g (`kb_supersede`, `kb_contradict`, …) without you asking each time, add: "When you find a stale claim, supersede it rather than proposing a contradicting one." +- Only a core set of `kb_*` tools is visible by default (`mcp.tool_profile: + minimal` in `.vouch/config.yaml`, or the `VOUCH_TOOL_PROFILE` env var). + Set it to `standard` or `full` to expose lifecycle/admin tools. diff --git a/adapters/codex/AGENTS.md.snippet b/adapters/codex/AGENTS.md.snippet new file mode 100644 index 00000000..593f97df --- /dev/null +++ b/adapters/codex/AGENTS.md.snippet @@ -0,0 +1,31 @@ +# Vouch — knowledge base + +This repo uses **vouch** for durable agent knowledge. The KB lives in +`.vouch/` and is reviewed in PRs like any other code. + +## How to remember things + +To preserve a fact, decision, or workflow across sessions: + +1. Register evidence: `kb_register_source` (or + `kb_register_source_from_path` for a file). +2. Propose a claim that cites it: `kb_propose_claim`. Every claim + MUST cite at least one source or evidence id. +3. For richer write-ups, propose pages: `kb_propose_page` with a + markdown body that references claims. + +You **cannot** write durable knowledge directly. Proposals land in +`.vouch/proposed/` and require human approval via `vouch approve`. +This is intentional. + +## How to read + +- `kb_search` for keyword search. +- `kb_context` to fill a working set for a task ("what does this KB know + about X?"). +- `kb_read_*` for specific ids. + +## Identity + +Set `VOUCH_AGENT=codex` in your env (or the MCP entry's `env:` block) so the +audit log can attribute writes to this host. diff --git a/adapters/codex/README.md b/adapters/codex/README.md index cfb324a4..5b8666c4 100644 --- a/adapters/codex/README.md +++ b/adapters/codex/README.md @@ -1,13 +1,86 @@ # Codex CLI adapter -Wires `vouch serve` into [OpenAI's Codex CLI][codex] as an MCP server. +Wires vouch into [OpenAI's Codex CLI][codex]: the MCP server, standing +`AGENTS.md` instructions, the vouch guided flows as skills, and +automatic session capture. [codex]: https://github.com/openai/codex -## Setup +## Install -Codex reads MCP server config from `~/.codex/config.toml`. Add a -`vouch` entry: +```bash +vouch install-mcp codex # everything (T1–T4) +vouch install-mcp codex --tier T1 # just the MCP wire +``` + +Everything lands in the *project*, never in `~/.codex` — the same +scope rule every adapter follows. Codex loads project-local `.codex/` +config for trusted projects, so trust the project when codex asks. + +What each tier adds (tiers stack): + +| Tier | File | What it does | +|---|---|---| +| T1 | `.codex/config.toml` | registers the `vouch` MCP server (`kb_search`, `kb_propose_claim`, …) with `VOUCH_AGENT=codex` for audit attribution | +| T2 | `AGENTS.md` | fenced snippet with the standing rules: recall first, all writes via proposals, review stays human | +| T3 | `.codex/skills/vouch-*/SKILL.md` | the nine vouch guided flows as project-local skills | +| T4 | `.codex/hooks.json` | `Stop` hook that auto-captures each session into a pending, review-gated summary | + +The install is idempotent and merge-safe: an existing +`.codex/config.toml` or `.codex/hooks.json` is deep-merged into (your +entries always win on conflict), an existing `AGENTS.md` gets the +snippet appended inside fence markers, and re-runs are a flat no-op. + +## Skills (T3) + +Codex discovers skills from `/.codex/skills/` in trusted +projects, so the flows (`vouch-recall`, `vouch-status`, +`vouch-resolve-issue`, `vouch-propose-from-pr`, plus the company-brain +set) surface in the session automatically — ask for one by name or let +codex pick them up from context. + +Why skills and not custom prompts: codex loads custom prompts only +from `~/.codex/prompts/` (user-global) and has deprecated them in +favour of skills. A project-scoped `vouch install-mcp` never touches +home-directory state, so skills are the surface vouch ships. If you +prefer slash-style prompts anyway, copy the installed +`.codex/skills/*/SKILL.md` bodies into `~/.codex/prompts/.md` +yourself. + +The skill bodies are identical to the claude-code slash commands +(enforced by a sync test), so the flows behave the same on every host. + +## Automatic session capture (T4) + +`.codex/hooks.json` registers a `Stop` hook that runs `vouch capture +ingest-codex --hook` when a turn completes. The handler reads the hook +payload, resolves the session's rollout file, and rolls it into ONE +pending session-summary proposal — the same review-gated summary a +claude-code session produces. Because `Stop` fires per turn, re-ingest +is idempotent: an unchanged session is a no-op, a session that grew +refreshes its pending proposal in place, and a proposal you've already +reviewed is never resurrected. + +Failure semantics match `capture observe`: the `--hook` mode exits 0 +no matter what, so a capture problem can never break your codex turn. +Nothing is auto-approved — review with `vouch review`. + +Past sessions can be ingested by hand too: + +```bash +vouch capture ingest-codex --latest # newest rollout for this project +vouch capture ingest-codex # a specific one +``` + +Why not codex's `notify` setting: codex honours `notify` only in +user-global config (`~/.codex/config.toml`), which a project-scoped +install never touches. If you prefer notify anyway, point it at a +wrapper that calls `vouch capture ingest-codex --hook` yourself. + +## Manual fallback + +No installer, or a user-global setup on purpose? Add the entry to +`~/.codex/config.toml` (or `/.codex/config.toml`) by hand: ```toml [mcp_servers.vouch] @@ -18,7 +91,8 @@ args = ["serve"] VOUCH_AGENT = "codex" ``` -Restart any running `codex` session. +Restart any running `codex` session, then confirm with +`codex mcp list`. ## Notes diff --git a/adapters/codex/hooks.json b/adapters/codex/hooks.json new file mode 100644 index 00000000..acaf8d04 --- /dev/null +++ b/adapters/codex/hooks.json @@ -0,0 +1,15 @@ +{ + "hooks": { + "Stop": [ + { + "hooks": [ + { + "type": "command", + "command": "vouch capture ingest-codex --hook", + "statusMessage": "vouch capture" + } + ] + } + ] + } +} diff --git a/adapters/codex/install.yaml b/adapters/codex/install.yaml index 7ba68b2f..9d4cb447 100644 --- a/adapters/codex/install.yaml +++ b/adapters/codex/install.yaml @@ -3,8 +3,49 @@ # Codex reads `/.codex/config.toml` (also `~/.codex/config.toml` for # user-global). We ship the project-local form so `vouch install-mcp codex` # doesn't touch home-directory state -- see issue #179 scope decision. +# +# `.codex/config.toml` is codex's *primary* config file (model, approval +# policy, other MCP servers), so T1 deep-merges into an existing one instead +# of skipping it -- see issue #384. Existing user values always win. host: codex pretty: OpenAI Codex CLI +fence: + begin: "" + end: "" tiers: T1: - - { src: config.toml, dst: .codex/config.toml } + - { src: config.toml, dst: .codex/config.toml, toml_merge: true } + # T2 = AGENTS.md fenced snippet (codex reads AGENTS.md for project + # instructions the way cursor does -- see issue #385). Kept in lockstep + # with adapters/cursor/AGENTS.md.snippet modulo the host name. + T2: + - { src: AGENTS.md.snippet, dst: AGENTS.md, fenced_append: true } + # T3 = the vouch guided flows as codex *skills* under the project-local + # `.codex/skills/` -- see issue #386. Scope decision recorded here per + # that ticket: codex custom prompts load only from ~/.codex/prompts/ + # (user-global) and are deprecated upstream in favour of skills, and the + # #179 rule forbids a project-scoped install from touching home-directory + # state, so prompts are out and project-local skills are in. The SKILL.md + # files are referenced from the openclaw mirror rather than duplicated; + # their bodies are sync-tested against the claude-code commands. + T3: + - { src: ../openclaw/skills/vouch-recall/SKILL.md, dst: .codex/skills/vouch-recall/SKILL.md } + - { src: ../openclaw/skills/vouch-status/SKILL.md, dst: .codex/skills/vouch-status/SKILL.md } + - { src: ../openclaw/skills/vouch-resolve-issue/SKILL.md, dst: .codex/skills/vouch-resolve-issue/SKILL.md } + - { src: ../openclaw/skills/vouch-propose-from-pr/SKILL.md, dst: .codex/skills/vouch-propose-from-pr/SKILL.md } + - { src: ../openclaw/skills/vouch-ask/SKILL.md, dst: .codex/skills/vouch-ask/SKILL.md } + - { src: ../openclaw/skills/vouch-remember/SKILL.md, dst: .codex/skills/vouch-remember/SKILL.md } + - { src: ../openclaw/skills/vouch-record/SKILL.md, dst: .codex/skills/vouch-record/SKILL.md } + - { src: ../openclaw/skills/vouch-followup/SKILL.md, dst: .codex/skills/vouch-followup/SKILL.md } + - { src: ../openclaw/skills/vouch-standup/SKILL.md, dst: .codex/skills/vouch-standup/SKILL.md } + # T4 = automatic session capture -- see issue #388. codex's hooks system + # fires Stop when a turn completes; the handler re-ingests the session's + # rollout idempotently (`vouch capture ingest-codex --hook` exits 0 even + # on failure, so capture can never break a codex turn), updating the + # session's single PENDING summary proposal as the session grows. + # hooks live in their own `.codex/hooks.json` file (project-local in + # trusted projects) and json_merge preserves any hooks the user already + # has. note: the legacy `notify` setting can't be used here -- codex only + # honours it in user-global config, which the #179 rule forbids touching. + T4: + - { src: hooks.json, dst: .codex/hooks.json, json_merge: true } diff --git a/docs/getting-started.md b/docs/getting-started.md index bb9ac038..b1003078 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -118,9 +118,17 @@ Add `-e VOUCH_AGENT=claude-code` to attribute the agent's proposals to it rather than your shell user. Confirm with `claude mcp list` (look for `vouch … ✓ Connected`). -Prefer a config file, or want the brain-first `CLAUDE.md`, slash commands, and -hooks too? Run `vouch install-mcp claude-code` — or drop this into `.mcp.json` -at the project root by hand: +Prefer a config file, or want the brain-first instructions, guided flows, and +capture hooks too? The installer ships the full tiered setup for either host: + +```bash +vouch install-mcp claude-code # .mcp.json + CLAUDE.md + slash commands + hooks +vouch install-mcp codex # .codex/config.toml + AGENTS.md + skills + capture +``` + +Both are idempotent and merge-safe — existing config files are deep-merged +into, never clobbered. Or drop this into `.mcp.json` at the project root by +hand: ```json { diff --git a/docs/superpowers/plans/2026-07-07-friendlier-mcp-surface.md b/docs/superpowers/plans/2026-07-07-friendlier-mcp-surface.md new file mode 100644 index 00000000..f68021cc --- /dev/null +++ b/docs/superpowers/plans/2026-07-07-friendlier-mcp-surface.md @@ -0,0 +1,930 @@ +# Friendlier MCP Surface Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make vouch feel as friendly as pmb by narrowing the MCP tool surface an agent sees (58 → ~8 by default), fusing retrieval by default, and injecting relevant context on every prompt — all without touching the review gate. + +**Architecture:** A new `mcp_profiles` module filters which `@mcp.tool()` tools the stdio server exposes (applied in `run_stdio`, not at import, so tests and the parity check still see all 58). `context._retrieve` is rewritten so `auto`/`hybrid` fuse embedding + FTS5 via the existing `rrf_fuse` instead of a first-non-empty waterfall, with a cheap near-duplicate drop. A tiny, always-safe `hooks` helper + a hidden `vouch context-hook` CLI command feed a Claude Code `UserPromptSubmit` hook so recall costs zero tool calls. + +**Tech Stack:** Python 3.11–3.13, `mcp.server.fastmcp.FastMCP`, click CLI, pytest, pydantic v2, yaml config, sqlite FTS5 + optional embeddings. + +## Global Constraints + +- **Review gate is untouched.** No task changes the write path (`proposals.py`, `lifecycle.py`, `storage.put_*`). Profiles change *exposure* only. +- **Protocol surface unchanged.** `capabilities.METHODS`, the JSONL `HANDLERS`, and the CLI keep all 58 methods. Only which MCP tools are *shown* narrows. +- **Default profile is `minimal`.** Resolution: `VOUCH_TOOL_PROFILE` env var > `config.yaml` `mcp.tool_profile` > `"minimal"`. Unknown → `"minimal"`. +- **CI gate must stay green:** `.venv/bin/python -m pytest tests/ -q --ignore=tests/embeddings`, `.venv/bin/python -m mypy src`, `.venv/bin/python -m ruff check src tests`. Every new function is fully type-annotated (mypy `src` is strict). +- **Commits:** conventional-commit format, summary ≤72 chars, lowercase body, **no `Co-Authored-By` trailer**. This repo's hooks reject `git commit -m` with heredocs and may reject `-m` outright — write the message to `/tmp/msg.txt` with the Write tool (never a heredoc) and use `git commit -F /tmp/msg.txt`. Stage files by name, never `git add -A`. +- **Branch:** `feat/friendlier-mcp-surface` (already created off `origin/main`; the spec is already committed on it). +- **Profile tool sets (dotted `METHODS` names), authoritative:** + - `minimal` (8): `kb.capabilities`, `kb.status`, `kb.context`, `kb.search`, `kb.read_page`, `kb.propose_claim`, `kb.propose_page`, `kb.list_pending` + - `standard` (minimal + 9 = 17): + `kb.approve`, `kb.reject`, `kb.supersede`, `kb.contradict`, `kb.confirm`, `kb.read_claim`, `kb.list_claims`, `kb.neighbors`, `kb.why` + - `full`: all of `capabilities.METHODS` (58) +- MCP tool names use `_`; `METHODS` uses `.`. Transform: `"kb.foo_bar"` ↔ `"kb_foo_bar"` via the substring after the first separator. + +--- + +### Task 1: MCP tool profiles + +**Files:** +- Create: `src/vouch/mcp_profiles.py` +- Modify: `src/vouch/server.py` (add import; apply profile inside `run_stdio` at `server.py:1011-1015`) +- Test: `tests/test_mcp_profiles.py` + +**Interfaces:** +- Produces: `mcp_profiles.PROFILES: dict[str, frozenset[str]]`, `mcp_profiles.DEFAULT_PROFILE: str`, `resolve_profile_name(config: dict | None = None) -> str`, `tool_names_for(name: str) -> set[str]`, `apply_tool_profile(mcp, name: str) -> list[str]` (returns removed underscore tool names, sorted). +- Consumes (Task 6): `compact_descriptions(mcp) -> int` is added to this module later. + +- [ ] **Step 1: Write the failing test** — create `tests/test_mcp_profiles.py`: + +```python +"""MCP tool profiles narrow the surface an agent sees (friendlier-mcp slice).""" + +from __future__ import annotations + +import pytest +from mcp.server.fastmcp import FastMCP + +from vouch import mcp_profiles +from vouch.capabilities import METHODS + +_MINIMAL_TOOLS = { + "kb_capabilities", "kb_status", "kb_context", "kb_search", + "kb_read_page", "kb_propose_claim", "kb_propose_page", "kb_list_pending", +} + + +def _make(name: str): + def fn(x: int = 0) -> int: + return x + fn.__name__ = name + return fn + + +def _fresh_mcp() -> FastMCP: + m = FastMCP("probe") + for method in METHODS: + m.tool()(_make("kb_" + method.split(".", 1)[1])) + return m + + +def test_full_profile_removes_nothing() -> None: + m = _fresh_mcp() + removed = mcp_profiles.apply_tool_profile(m, "full") + assert removed == [] + assert len(m._tool_manager._tools) == len(METHODS) + + +def test_minimal_exposes_core_only() -> None: + m = _fresh_mcp() + mcp_profiles.apply_tool_profile(m, "minimal") + assert set(m._tool_manager._tools) == _MINIMAL_TOOLS + + +def test_standard_is_superset_of_minimal() -> None: + assert mcp_profiles.PROFILES["minimal"] <= mcp_profiles.PROFILES["standard"] + + +def test_every_profile_is_subset_of_methods() -> None: + allm = set(METHODS) + for name, methods in mcp_profiles.PROFILES.items(): + assert methods <= allm, f"{name} references non-methods: {methods - allm}" + assert mcp_profiles.PROFILES["full"] == allm + + +def test_resolve_env_beats_config(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("VOUCH_TOOL_PROFILE", "full") + assert mcp_profiles.resolve_profile_name({"mcp": {"tool_profile": "minimal"}}) == "full" + + +def test_resolve_default_is_minimal(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.delenv("VOUCH_TOOL_PROFILE", raising=False) + assert mcp_profiles.resolve_profile_name(None) == "minimal" + + +def test_unknown_profile_falls_back_to_minimal(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("VOUCH_TOOL_PROFILE", "bogus") + assert mcp_profiles.resolve_profile_name(None) == "minimal" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_mcp_profiles.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'vouch.mcp_profiles'`. + +- [ ] **Step 3: Write minimal implementation** — create `src/vouch/mcp_profiles.py`: + +```python +"""MCP tool profiles — narrow the tool surface an agent sees by default. + +vouch exposes 58 kb.* methods. Handing all of them to an agent every turn is +the main first-touch friendliness cost (the closest competitor, pmb, exposes +~10 by default and hides the rest behind a profile flag). Profiles control +*exposure* only: the JSONL and CLI surfaces, the protocol method list +(capabilities.METHODS), and the review gate are unchanged. + +Resolution order: VOUCH_TOOL_PROFILE env var > config.yaml `mcp.tool_profile` +> "minimal" (the default). Unknown names fall back to "minimal". +""" + +from __future__ import annotations + +import os +from typing import Any + +from .capabilities import METHODS + +_MINIMAL: frozenset[str] = frozenset({ + "kb.capabilities", + "kb.status", + "kb.context", + "kb.search", + "kb.read_page", + "kb.propose_claim", + "kb.propose_page", + "kb.list_pending", +}) + +_STANDARD: frozenset[str] = _MINIMAL | frozenset({ + "kb.approve", + "kb.reject", + "kb.supersede", + "kb.contradict", + "kb.confirm", + "kb.read_claim", + "kb.list_claims", + "kb.neighbors", + "kb.why", +}) + +PROFILES: dict[str, frozenset[str]] = { + "minimal": _MINIMAL, + "standard": _STANDARD, + "full": frozenset(METHODS), +} + +DEFAULT_PROFILE = "minimal" + + +def _tool_name(method: str) -> str: + """`"kb.propose_claim"` -> `"kb_propose_claim"` (MCP tool names use `_`).""" + return "kb_" + method.split(".", 1)[1] + + +def resolve_profile_name(config: dict[str, Any] | None = None) -> str: + """Pick the active profile from env > config > default.""" + raw = os.environ.get("VOUCH_TOOL_PROFILE") + if not raw and config: + raw = config.get("mcp", {}).get("tool_profile") + name = str(raw).strip().lower() if raw else DEFAULT_PROFILE + return name if name in PROFILES else DEFAULT_PROFILE + + +def tool_names_for(name: str) -> set[str]: + """The MCP (underscore) tool names exposed by profile `name`.""" + return {_tool_name(m) for m in PROFILES.get(name, PROFILES[DEFAULT_PROFILE])} + + +def apply_tool_profile(mcp: Any, name: str) -> list[str]: + """Remove every registered `kb_*` MCP tool not in profile `name`. + + Returns the sorted list of removed tool names. Idempotent. `full` removes + nothing. Only `kb_*` tools are touched, so trust/diagnostic tools + registered elsewhere are never dropped. + """ + keep = tool_names_for(name) + tools = mcp._tool_manager._tools + removed: list[str] = [] + for tool_name in list(tools.keys()): + if tool_name.startswith("kb_") and tool_name not in keep: + tools.pop(tool_name, None) + removed.append(tool_name) + return sorted(removed) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_mcp_profiles.py -q` +Expected: PASS (7 tests). + +- [ ] **Step 5: Wire the profile into the stdio server** — modify `src/vouch/server.py`. Add to the imports near the top (with the other `from .` imports, e.g. after `from .synthesize import synthesize` at line 56): + +```python +from . import mcp_profiles +``` + +Then replace `run_stdio` (currently `server.py:1011-1015`): + +```python +def run_stdio() -> None: + """Entry point used by `vouch serve`.""" + configure_logging() + trust_mod.set_stdio_default(trust_mod.MCP_STDIO) + try: + cfg: dict[str, Any] | None = _load_cfg(_store()) + except Exception: + cfg = None + mcp_profiles.apply_tool_profile(mcp, mcp_profiles.resolve_profile_name(cfg)) + mcp.run() +``` + +(`Any` is already imported in `server.py`; `_load_cfg` is at `server.py:204` and `_store` at `server.py:61`.) + +- [ ] **Step 6: Verify server still imports and mypy/ruff pass** + +Run: `.venv/bin/python -c "import vouch.server"` → Expected: no output, exit 0. +Run: `.venv/bin/python -m mypy src/vouch/mcp_profiles.py src/vouch/server.py` → Expected: `Success`. +Run: `.venv/bin/python -m ruff check src/vouch/mcp_profiles.py src/vouch/server.py` → Expected: `All checks passed!`. + +- [ ] **Step 7: Commit** + +Write `/tmp/msg.txt`: +``` +feat(mcp): add tool profiles, expose a minimal surface by default + +agents saw all 58 kb.* tools every turn — the main first-touch friendliness +gap vs pmb, which exposes ~10 by default. add a profile layer applied in +run_stdio: minimal (8 core tools) by default, standard (16), or full (58), +selected by VOUCH_TOOL_PROFILE / config mcp.tool_profile. exposure only — the +protocol surface, jsonl/cli, and the review gate are unchanged. +``` +Run: +```bash +git add src/vouch/mcp_profiles.py src/vouch/server.py tests/test_mcp_profiles.py +git commit -F /tmp/msg.txt +``` + +--- + +### Task 2: Enforce true MCP↔METHODS parity + +**Files:** +- Modify: `tests/test_capabilities.py` + +**Interfaces:** +- Consumes: `vouch.server.mcp` (the module-level, unfiltered `FastMCP`), `vouch.capabilities.METHODS`. + +**Why:** `test_capabilities.py` today only checks `capabilities.METHODS == JSONL HANDLERS`. The 58 MCP tools were never compared to `METHODS`, so MCP drift passed CI (the CLAUDE.md "all three surfaces" claim was really 2-of-3). Importing `vouch.server` registers all tools but does **not** apply a profile (that happens only in `run_stdio`), so this sees the full surface. + +- [ ] **Step 1: Write the failing test** — append to `tests/test_capabilities.py`: + +```python +def test_mcp_tools_match_methods() -> None: + """Every MCP kb_* tool maps to a capabilities method and vice-versa. + + Closes the MCP half of the 3-surface parity invariant that the JSONL + check above did not cover. Uses the unfiltered server object (profiles + apply only in run_stdio). + """ + from vouch.server import mcp + + tool_names = {n for n in mcp._tool_manager._tools if n.startswith("kb_")} + as_methods = {"kb." + n.split("_", 1)[1] for n in tool_names} + declared = set(capabilities.METHODS) + assert as_methods == declared, ( + f"mcp/methods mismatch: " + f"missing tools={declared - as_methods}, " + f"undeclared tools={as_methods - declared}" + ) +``` + +- [ ] **Step 2: Run test to verify it passes or reveals real drift** + +Run: `.venv/bin/python -m pytest tests/test_capabilities.py -q` +Expected: PASS if the 58 tools already align with `METHODS`. If it FAILS, the message names the exact drift — fix `capabilities.METHODS` or the tool registration in `server.py` until it passes (this is a real bug the test just caught, not a test error). Do not weaken the assertion. + +- [ ] **Step 3: Commit** + +Write `/tmp/msg.txt`: +``` +test(capabilities): assert mcp tool set matches the method list + +the parity test only compared capabilities.METHODS to the jsonl handlers; +the 58 mcp tools were never checked, so mcp drift passed ci. enumerate the +unfiltered server tools and assert they equal METHODS — the real 3-surface +check for the mcp side. +``` +Run: +```bash +git add tests/test_capabilities.py +git commit -F /tmp/msg.txt +``` + +--- + +### Task 3: Fuse retrieval by default + +**Files:** +- Modify: `src/vouch/context.py` (`_VALID_BACKENDS` at line 45; rewrite `_retrieve` body at lines 90-120; add a top-level import) +- Modify: `src/vouch/storage.py` (default backend at line 96) +- Modify: `tests/test_retrieval_backend.py` (update the two `auto` tests whose behavior intentionally changes; add a hybrid-fusion test) + +**Interfaces:** +- Consumes: `vouch.embeddings.fusion.rrf_fuse(a, b, *, limit=10, k=60)` where `a`/`b` are `list[tuple[str, str, str, float]]` (kind, id, summary, score). +- Produces: `_retrieve` now tags fused hits with backend `"hybrid"`; `auto` and `hybrid` both fuse embedding + FTS5. + +- [ ] **Step 1: Update the two tests whose behavior changes + add the fusion test.** In `tests/test_retrieval_backend.py`, replace `test_backend_auto_prefers_embedding` (lines 82-89) and `test_unset_backend_defaults_to_auto` (lines 92-101) with: + +```python +def test_backend_auto_now_fuses( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + """`auto` no longer waterfalls embedding-first; it fuses embedding + fts5 + (RRF) and tags hits `hybrid`.""" + _force_semantic_hit(monkeypatch) + _set_backend(store, "auto") + pack = context.build_context_pack(store, query="JWT") + assert pack["items"] + assert _backends(pack) == {"hybrid"} + + +def test_unset_backend_fuses( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + """A config with no retrieval.backend behaves like fused `auto`.""" + _force_semantic_hit(monkeypatch) + cfg = yaml.safe_load(store.config_path.read_text()) + cfg.get("retrieval", {}).pop("backend", None) + store.config_path.write_text(yaml.safe_dump(cfg)) + pack = context.build_context_pack(store, query="JWT") + assert _backends(pack) == {"hybrid"} + + +def test_backend_hybrid_merges_semantic_and_lexical( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + """`hybrid` returns the union of both retrievers, not first-non-empty.""" + src = store.put_source(b"e2") + store.put_claim(Claim(id="c2", text="OAuth refresh flow", evidence=[src.id])) + health.rebuild_index(store) + monkeypatch.setattr( + context.index_db, "search_semantic", + lambda *a, **k: [("claim", "c1", "JWT token rotation", 0.99)], + ) + monkeypatch.setattr( + context.index_db, "search", + lambda *a, **k: [("claim", "c2", "OAuth refresh flow", 0.88)], + ) + _set_backend(store, "hybrid") + pack = context.build_context_pack(store, query="auth") + assert {item["id"] for item in pack["items"]} == {"c1", "c2"} + assert _backends(pack) == {"hybrid"} +``` + +- [ ] **Step 2: Run tests to verify the new ones fail** + +Run: `.venv/bin/python -m pytest tests/test_retrieval_backend.py -q` +Expected: FAIL — `test_backend_auto_now_fuses` / `test_unset_backend_fuses` / `test_backend_hybrid_merges_semantic_and_lexical` fail because `hybrid` isn't accepted yet and `auto` still waterfalls (backend is `embedding`, not `hybrid`). + +- [ ] **Step 3: Implement fusion.** In `src/vouch/context.py`: + +(a) Add near the top imports (after the existing `from . import` block): +```python +from .embeddings.fusion import rrf_fuse +``` +(`fusion.py` is pure-Python with no heavy deps, so a top-level import is safe under the base install.) + +(b) Change `_VALID_BACKENDS` (line 45): +```python +_VALID_BACKENDS = ("auto", "hybrid", "embedding", "fts5", "substring") +``` + +(c) Replace the body of `_retrieve` from line 90 (`backend = _configured_backend(store)`) through line 120 (the final substring `return`) with: +```python + backend = _configured_backend(store) + fetch_limit = scoped_fetch_limit(limit, viewer) + + if backend in ("auto", "hybrid"): + sem = index_db.search_semantic(store.kb_dir, query, limit=fetch_limit) + try: + lex = index_db.search(store.kb_dir, query, limit=fetch_limit) + except sqlite3.Error: + lex = [] + fused = rrf_fuse(sem, lex, limit=fetch_limit) + if fused: + filtered = filter_hits(store, fused, viewer, limit=limit) + return [(k, i, s, sc, "hybrid") for k, i, s, sc in filtered] + # both retrievers empty -> fall through to the substring scan below. + + if backend == "embedding": + raw = index_db.search_semantic(store.kb_dir, query, limit=fetch_limit) + if raw: + filtered = filter_hits(store, raw, viewer, limit=limit) + return [(k, i, s, sc, "embedding") for k, i, s, sc in filtered] + return [] + + if backend == "fts5": + try: + hits = index_db.search(store.kb_dir, query, limit=fetch_limit) + if hits: + filtered = filter_hits(store, hits, viewer, limit=limit) + return [(k, i, s, sc, "fts5") for k, i, s, sc in filtered] + except sqlite3.Error: + pass + return [] + + substring_hits = store.search_substring(query, limit=fetch_limit) + filtered = filter_hits(store, substring_hits, viewer, limit=limit) + return [(k, i, s, sc, "substring") for k, i, s, sc in filtered] +``` + +(d) In `src/vouch/storage.py` line 96, change the init default so new KBs say `hybrid` explicitly: +```python + "backend": "hybrid", +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `.venv/bin/python -m pytest tests/test_retrieval_backend.py -q` +Expected: PASS. `test_backend_fts5_skips_embedding`, `test_backend_embedding_is_recognized`, `test_backend_substring_only` still pass (explicit pins unchanged). + +- [ ] **Step 5: Guard against retrieval-quality regression + mypy/ruff** + +Run: `.venv/bin/python -m pytest tests/test_context.py tests/test_index.py -q` → Expected: PASS. +Run: `.venv/bin/python -m vouch eval recall eval/queries.jsonl --kb eval/fixture-kb --baseline eval/baseline.json --max-regression 0.05` (the exact args `eval.yml` uses; if the CLI flags differ, run `.venv/bin/vouch eval recall --help` and match them) → Expected: no regression beyond 0.05. +Run: `.venv/bin/python -m mypy src/vouch/context.py src/vouch/storage.py` → Expected: `Success`. +Run: `.venv/bin/python -m ruff check src/vouch/context.py src/vouch/storage.py` → Expected: `All checks passed!`. + +- [ ] **Step 6: Commit** + +Write `/tmp/msg.txt`: +``` +feat(retrieval): fuse embedding + fts5 by default instead of a waterfall + +_retrieve tried embedding, then fts5, then substring, returning the first +non-empty list — so lexical and semantic hits never combined. auto and +hybrid now fuse both retrievers with the already-built rrf_fuse and tag hits +"hybrid"; explicit embedding/fts5/substring pins are unchanged. existing KBs +(config says "auto") benefit with no migration. +``` +Run: +```bash +git add src/vouch/context.py src/vouch/storage.py tests/test_retrieval_backend.py +git commit -F /tmp/msg.txt +``` + +--- + +### Task 4: Drop near-duplicate context items + +**Files:** +- Modify: `src/vouch/context.py` (add `_jaccard` + `_dedupe_near_duplicates`; call it in `build_context_pack` after graph expansion, ~line 256) +- Test: `tests/test_retrieval_backend.py` (add one test; reuses the existing `store` fixture) + +**Interfaces:** +- Produces: `build_context_pack` drops lower-scored items whose summary is near-identical (token-set Jaccard ≥ 0.85) to a higher-scored kept item. + +- [ ] **Step 1: Write the failing test** — append to `tests/test_retrieval_backend.py`: + +```python +def test_near_duplicate_summaries_are_dropped( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + """An agent should not see the same fact twice.""" + src = store.put_source(b"z") + store.put_claim(Claim( + id="d1", text="the cache uses redis with a 60 second ttl", evidence=[src.id])) + store.put_claim(Claim( + id="d2", text="the cache uses redis with a 60 second ttl now", evidence=[src.id])) + health.rebuild_index(store) + monkeypatch.setattr( + context.index_db, "search_semantic", + lambda *a, **k: [ + ("claim", "d1", "the cache uses redis with a 60 second ttl", 0.90), + ("claim", "d2", "the cache uses redis with a 60 second ttl now", 0.89), + ], + ) + monkeypatch.setattr(context.index_db, "search", lambda *a, **k: []) + _set_backend(store, "hybrid") + pack = context.build_context_pack(store, query="cache") + assert {item["id"] for item in pack["items"]} == {"d1"} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_retrieval_backend.py::test_near_duplicate_summaries_are_dropped -q` +Expected: FAIL — both `d1` and `d2` present. + +- [ ] **Step 3: Implement the dedupe pass.** In `src/vouch/context.py`, add these helpers above `build_context_pack` (e.g. after `_append_graph_neighbors`, ~line 198): + +```python +def _jaccard(a: set[str], b: set[str]) -> float: + if not a or not b: + return 0.0 + return len(a & b) / len(a | b) + + +def _dedupe_near_duplicates(items: list[ContextItem]) -> list[ContextItem]: + """Drop later items whose summary is near-identical to a kept one. + + Cheap greedy pass (token-set Jaccard >= 0.85 over the first 40 tokens). + `items` must arrive in descending-score order so the higher-scored + duplicate is the one kept. + """ + kept: list[ContextItem] = [] + kept_tokens: list[set[str]] = [] + for it in items: + toks = set(it.summary.lower().split()[:40]) + if any(_jaccard(toks, seen) >= 0.85 for seen in kept_tokens): + continue + kept.append(it) + kept_tokens.append(toks) + return kept +``` + +Then in `build_context_pack`, immediately after the `if expand_graph:` block closes (after line 256, before the `failed: list[str] = []` line at 257) insert: + +```python + items = _dedupe_near_duplicates(items) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_retrieval_backend.py -q` → Expected: PASS (all, including the earlier fusion tests — single non-duplicate hits are unaffected). + +- [ ] **Step 5: mypy/ruff** + +Run: `.venv/bin/python -m mypy src/vouch/context.py` → Expected: `Success`. +Run: `.venv/bin/python -m ruff check src/vouch/context.py` → Expected: `All checks passed!`. + +- [ ] **Step 6: Commit** + +Write `/tmp/msg.txt`: +``` +feat(retrieval): drop near-duplicate items from the context pack + +a fused pack could surface the same fact from two claims. add a cheap greedy +jaccard pass (>=0.85 over the first 40 tokens) that keeps the highest-scored +of a near-duplicate cluster, so an agent never reads the same thing twice. +``` +Run: +```bash +git add src/vouch/context.py tests/test_retrieval_backend.py +git commit -F /tmp/msg.txt +``` + +--- + +### Task 5: Per-prompt auto-recall hook + +**Files:** +- Create: `src/vouch/hooks.py` +- Modify: `src/vouch/cli.py` (add a hidden `context-hook` command near the `context` command at `cli.py:2272`) +- Modify: `adapters/claude-code/.claude/settings.json` (add a `UserPromptSubmit` hook) +- Test: `tests/test_hooks.py` + +**Interfaces:** +- Produces: `hooks.build_claude_prompt_hook(store: KBStore, stdin_text: str) -> str` — returns the JSON envelope to inject, or `""` for nothing. Never raises. +- Consumes: `context.build_context_pack`, `cli._load_store`. + +- [ ] **Step 1: Write the failing test** — create `tests/test_hooks.py`: + +```python +"""The per-prompt hook injects relevant KB context with zero tool calls.""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from vouch import context, health, hooks +from vouch.models import Claim +from vouch.storage import KBStore + + +@pytest.fixture +def store(tmp_path: Path) -> KBStore: + s = KBStore.init(tmp_path) + src = s.put_source(b"e") + s.put_claim(Claim(id="c1", text="deploys run on tuesdays via ci", evidence=[src.id])) + health.rebuild_index(s) + return s + + +def _force_hit(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr( + context.index_db, "search_semantic", + lambda *a, **k: [("claim", "c1", "deploys run on tuesdays via ci", 0.9)], + ) + monkeypatch.setattr(context.index_db, "search", lambda *a, **k: []) + + +def test_empty_prompt_injects_nothing(store: KBStore) -> None: + assert hooks.build_claude_prompt_hook(store, json.dumps({"prompt": ""})) == "" + assert hooks.build_claude_prompt_hook(store, "") == "" + + +def test_relevant_prompt_yields_additional_context( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + _force_hit(monkeypatch) + out = hooks.build_claude_prompt_hook(store, json.dumps({"prompt": "when do deploys run"})) + env = json.loads(out) + assert env["hookSpecificOutput"]["hookEventName"] == "UserPromptSubmit" + assert "tuesdays" in env["hookSpecificOutput"]["additionalContext"] + + +def test_raw_non_json_stdin_is_tolerated( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + _force_hit(monkeypatch) + out = hooks.build_claude_prompt_hook(store, "when do deploys run") + assert "tuesdays" in json.loads(out)["hookSpecificOutput"]["additionalContext"] + + +def test_no_hits_injects_nothing( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setattr(context.index_db, "search_semantic", lambda *a, **k: []) + monkeypatch.setattr(context.index_db, "search", lambda *a, **k: []) + assert hooks.build_claude_prompt_hook(store, json.dumps({"prompt": "zzznomatch"})) == "" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_hooks.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'vouch.hooks'`. + +- [ ] **Step 3: Implement the helper** — create `src/vouch/hooks.py`: + +```python +"""Host-hook helpers: translate an agent host's prompt hook into KB context. + +Claude Code's UserPromptSubmit hook passes a JSON payload on stdin and injects +whatever the hook prints (as an `additionalContext` envelope) before the model +runs. `build_claude_prompt_hook` turns that payload into a compact, relevant +context block drawn from *approved* KB knowledge — so recall costs the agent +zero tool calls. It never raises: on any problem it returns "" (inject +nothing), so a hook failure can never block the user's turn. +""" + +from __future__ import annotations + +import json +from typing import Any + +from .context import build_context_pack +from .storage import KBStore + +_MAX_ITEMS = 8 +_MAX_CHARS = 2000 + + +def _render(pack: dict[str, Any]) -> str: + lines: list[str] = [] + for item in pack.get("items", []): + summary = str(item.get("summary", "")).strip() + if not summary: + continue + cites = item.get("citations") or [] + suffix = f" [{', '.join(cites)}]" if cites else "" + lines.append(f"- {summary}{suffix}") + return "\n".join(lines) + + +def build_claude_prompt_hook(store: KBStore, stdin_text: str) -> str: + """Return the stdout a host should inject for this prompt, or "" for none.""" + try: + payload = json.loads(stdin_text) if stdin_text.strip() else {} + except json.JSONDecodeError: + payload = {"prompt": stdin_text} + prompt = str(payload.get("prompt", "")).strip() + if not prompt: + return "" + try: + pack = build_context_pack( + store, query=prompt, limit=_MAX_ITEMS, max_chars=_MAX_CHARS, + ) + except Exception: + return "" + body = _render(pack) if isinstance(pack, dict) else "" + if not body: + return "" + block = ( + "Relevant knowledge from the project's vouch KB " + "(approved & cited — consider it before answering):\n" + body + ) + return json.dumps({ + "hookSpecificOutput": { + "hookEventName": "UserPromptSubmit", + "additionalContext": block, + } + }) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_hooks.py -q` → Expected: PASS (4 tests). + +- [ ] **Step 5: Add the hidden CLI command.** In `src/vouch/cli.py`, immediately after the `context` command (after `_emit_json(pack)` at `cli.py:2299`, before the `synthesize` command at 2302) add: + +```python +@cli.command(name="context-hook", hidden=True) +def context_hook() -> None: + """Emit relevant KB context for a host UserPromptSubmit hook (reads stdin). + + Wired by the claude-code adapter; not meant to be run by hand. Reads the + host's JSON hook payload on stdin, prints an additionalContext envelope, + and always exits 0 so it can never block a turn. + """ + import sys + + from . import hooks + + stdin_text = sys.stdin.read() + try: + out = hooks.build_claude_prompt_hook(_load_store(), stdin_text) + except Exception: + out = "" + if out: + click.echo(out) +``` + +- [ ] **Step 6: Verify the command runs end-to-end** + +Run: `printf '{"prompt":"anything"}' | .venv/bin/vouch context-hook` from inside a KB dir (or `cd eval/fixture-kb && printf '{"prompt":"jwt"}' | ../../.venv/bin/vouch context-hook`). +Expected: either empty output (no hits) or a single line of JSON containing `"hookEventName": "UserPromptSubmit"`. Never a traceback, always exit 0 (`echo $?` → `0`). + +- [ ] **Step 7: Wire the hook into the claude-code adapter.** In `adapters/claude-code/.claude/settings.json`, add a `UserPromptSubmit` block inside `"hooks"` (after the `SessionStart` block, before `PostToolUse`). The file's `"hooks"` object becomes: + +```json + "hooks": { + "SessionStart": [ + { + "comment": "finalize old buffers from previous sessions; current session will be finalized here too on next session start (fallback: windowclose event not yet supported by claude-code extension)", + "matcher": "*", + "hooks": [ + { "type": "command", "command": "vouch capture finalize-all || true" }, + { "type": "command", "command": "vouch status --json || true" }, + { "type": "command", "command": "vouch capture banner || true" }, + { "type": "command", "command": "vouch recall || true" } + ] + } + ], + "UserPromptSubmit": [ + { + "comment": "inject KB context relevant to THIS prompt (per-prompt auto-recall, 0 tool calls); never blocks the turn", + "matcher": "*", + "hooks": [ + { "type": "command", "command": "vouch context-hook || true" } + ] + } + ], + "PostToolUse": [ + { + "matcher": "*", + "hooks": [ + { "type": "command", "command": "vouch capture observe || true" } + ] + } + ], + "SessionEnd": [ + { + "matcher": "*", + "hooks": [ + { "type": "command", "command": "vouch capture finalize || true" } + ] + } + ] + } +``` + +(`kb_context` is already in the adapter's permission allowlist, and the hook uses the CLI, so no permission change is needed. `vouch recall` stays at SessionStart — it serves the session banner, a different purpose from per-prompt recall.) + +- [ ] **Step 8: Verify the adapter JSON stays valid + install-merge test passes** + +Run: `.venv/bin/python -c "import json; json.load(open('adapters/claude-code/.claude/settings.json'))"` → Expected: no output, exit 0. +Run: `.venv/bin/python -m pytest tests/test_install_adapter.py -q` → Expected: PASS. + +- [ ] **Step 9: Commit** + +Write `/tmp/msg.txt`: +``` +feat(hooks): inject per-prompt kb context via a userpromptsubmit hook + +recall used to be either a session-start firehose or an explicit tool call. +add a pure, never-raising helper + a hidden `vouch context-hook` command that +reads the host's prompt payload on stdin and prints an additionalContext +envelope, and wire it into the claude-code adapter's UserPromptSubmit hook — +so relevant, approved knowledge is injected every turn with zero tool calls. +``` +Run: +```bash +git add src/vouch/hooks.py src/vouch/cli.py adapters/claude-code/.claude/settings.json tests/test_hooks.py +git commit -F /tmp/msg.txt +``` + +--- + +### Task 6: Compact tool descriptions under non-full profiles + +**Files:** +- Modify: `src/vouch/mcp_profiles.py` (add `compact_descriptions`) +- Modify: `src/vouch/server.py` (call it in `run_stdio` when profile != `full`) +- Test: `tests/test_mcp_profiles.py` (add one test) + +**Interfaces:** +- Produces: `mcp_profiles.compact_descriptions(mcp) -> int` — trims each `kb_*` tool's description to its first line; returns count changed. + +- [ ] **Step 1: Write the failing test** — append to `tests/test_mcp_profiles.py`: + +```python +def test_compact_descriptions_trims_to_first_line() -> None: + m = FastMCP("probe") + + def kb_thing(x: int = 0) -> int: + """First line. + + Second paragraph with lots of detail the agent does not need. + """ + return x + + m.tool()(kb_thing) + changed = mcp_profiles.compact_descriptions(m) + assert changed == 1 + assert m._tool_manager._tools["kb_thing"].description == "First line." +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_mcp_profiles.py::test_compact_descriptions_trims_to_first_line -q` +Expected: FAIL — `AttributeError: module 'vouch.mcp_profiles' has no attribute 'compact_descriptions'`. + +- [ ] **Step 3: Implement.** Append to `src/vouch/mcp_profiles.py`: + +```python +def compact_descriptions(mcp: Any) -> int: + """Trim each kb_ tool's description to its first line to save context. + + Full docstrings are only needed under the `full` profile; the first line + is enough for an agent choosing a tool. Returns the number changed. + """ + changed = 0 + for tool in mcp._tool_manager._tools.values(): + if not tool.name.startswith("kb_"): + continue + desc = tool.description or "" + first = desc.strip().split("\n", 1)[0].strip() + if first and first != desc: + try: + tool.description = first + except (AttributeError, TypeError): + # pydantic frozen-model fallback + object.__setattr__(tool, "description", first) + changed += 1 + return changed +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_mcp_profiles.py -q` → Expected: PASS (8 tests). + +- [ ] **Step 5: Call it from the server.** In `src/vouch/server.py` `run_stdio`, insert after the `apply_tool_profile(...)` line and before `mcp.run()`: + +```python + if mcp_profiles.resolve_profile_name(cfg) != "full": + mcp_profiles.compact_descriptions(mcp) +``` + +(Resolve is cheap and pure; calling it twice is fine. Alternatively hoist the resolved name into a local `profile` var and reuse — either is acceptable.) + +- [ ] **Step 6: mypy/ruff + import check** + +Run: `.venv/bin/python -c "import vouch.server"` → Expected: exit 0. +Run: `.venv/bin/python -m mypy src/vouch/mcp_profiles.py src/vouch/server.py` → Expected: `Success`. +Run: `.venv/bin/python -m ruff check src/vouch/mcp_profiles.py src/vouch/server.py` → Expected: `All checks passed!`. + +- [ ] **Step 7: Commit** + +Write `/tmp/msg.txt`: +``` +feat(mcp): serve one-line tool descriptions under non-full profiles + +full docstrings for every exposed tool are paid on every turn. under minimal +and standard, trim each tool's description to its first line (full keeps the +complete docstrings), cutting the per-turn context cost. +``` +Run: +```bash +git add src/vouch/mcp_profiles.py src/vouch/server.py tests/test_mcp_profiles.py +git commit -F /tmp/msg.txt +``` + +--- + +## Final verification + +Run the full CI gate exactly as `.github/workflows/ci.yml` does, plus a manual end-to-end (per the "verify the shipped diff" rule — exercise the real behavior, not just unit tests): + +- [ ] `.venv/bin/python -m pytest tests/ -q --ignore=tests/embeddings` → all pass. +- [ ] `.venv/bin/python -m mypy src` → `Success`. +- [ ] `.venv/bin/python -m ruff check src tests` → `All checks passed!`. +- [ ] **Surface, minimal (default):** in a KB dir, run the server and count tools — + `.venv/bin/python -c "import vouch.server as s, vouch.mcp_profiles as p; p.apply_tool_profile(s.mcp, 'minimal'); print(sorted(n for n in s.mcp._tool_manager._tools if n.startswith('kb_')))"` → exactly the 8 minimal tools. +- [ ] **Surface, full override:** same one-liner with `'full'` → all 58 present. +- [ ] **Auto-recall e2e:** `cd eval/fixture-kb && printf '{"prompt":"how is rate limiting done"}' | ../../.venv/bin/vouch context-hook` → one JSON line with `additionalContext` drawn from the fixture claims; `echo $?` → `0`. +- [ ] **Docs (light):** add one line documenting `VOUCH_TOOL_PROFILE` / `mcp.tool_profile` (default `minimal`, values `minimal|standard|full`) to `mintlify/reference/` (the config/MCP reference) and the claude-code guide. Commit as `docs(mcp): document tool profiles`. + +Then hand back to the user for review before any push (this branch is off `main`; pushing is a separate, explicit step). diff --git a/docs/superpowers/specs/2026-05-25-vouch-diff-design.md b/docs/superpowers/specs/2026-05-25-vouch-diff-design.md index f5bb4ea0..5bd1f862 100644 --- a/docs/superpowers/specs/2026-05-25-vouch-diff-design.md +++ b/docs/superpowers/specs/2026-05-25-vouch-diff-design.md @@ -23,7 +23,14 @@ field. Read-only: no writes, no proposals, no audit events. `last_confirmed_at`, `approved_by`). - **Line-diff the long text.** `claim.text` / `page.body` render as a `difflib` unified diff; everything else as `field: old → new`. -- **CLI-only.** Read-only inspection; does not touch the `kb.*` capability set. +- **Full `kb.*` parity.** Registered as `kb.diff` at all four sites (MCP tool, + JSONL handler, `capabilities.METHODS`, CLI) like any other read method — + see "MCP/JSONL parity" under Non-goals below for the superseded original + call on this. +- **Omitted `new_id` resolves via `superseded_by`.** For a claim, `new_id` is + optional; when omitted it resolves to `old_claim.superseded_by`, erroring + clearly if that's unset. Pages have no successor pointer, so `new_id` is + required for a page. ## Components — `src/vouch/diff.py` @@ -37,7 +44,10 @@ Raised for unknown ids and mismatched kinds. `kind: str, old_id: str, new_id: str, changes: list[FieldChange], text_diff: list[str]`. -### `diff_artifacts(store, old_id, new_id) -> ArtifactDiff` +### `diff_artifacts(store, old_id, new_id=None) -> ArtifactDiff` +- **`new_id` resolution:** if omitted, `old_id` must resolve to a claim with + `superseded_by` set — that becomes `new_id`. A page, or a claim without a + successor, raises `DiffError` naming the id. - **Kind resolution:** try `store.get_claim` on both ids → both succeed ⇒ `kind="claim"`. Otherwise try `store.get_page` on both → `kind="page"`. If an id resolves to neither, raise `DiffError("unknown artifact: ")`. If one is @@ -55,7 +65,7 @@ Field sets (long text field rendered as `text_diff`, the rest as changes): - **Page** — body *(diff)*; title, type, status, claims, entities, sources, tags. -## CLI — `vouch diff OLD NEW [--json]` +## CLI — `vouch diff OLD [NEW] [--json]` Follows existing patterns (`_load_store`, `_cli_errors`, `_emit_json`). @@ -74,6 +84,20 @@ diff claim → - `--json` → `_emit_json` of the `ArtifactDiff` as a dict. - No differences → prints `no differences`. +## `kb.diff` — MCP + JSONL + +Same read as the CLI, exposed for agents: + +- **MCP** `kb_diff(old_id, new_id=None) -> dict` in `server.py`, next to + `kb_read_claim`/`kb_read_page` in the unrestricted-read section. +- **JSONL** `_h_diff` reads `params["old_id"]` (required) and + `params["new_id"]` (optional) — `kb.diff` in `HANDLERS`. +- **capabilities** `kb.diff` in `METHODS`, next to `kb.read_relation`. +- Both return `dataclasses.asdict(ArtifactDiff)`. +- Unrestricted like the other by-id read tools (`kb_read_claim`, + `kb_read_page`) — no `ViewerContext`/scope filtering, since resolving a + *specific known id* carries the same exposure either way. + ## Error handling - Unknown id (neither claim nor page) → `DiffError` → clean CLI `Error:` line. @@ -90,6 +114,14 @@ diff claim → ## Non-goals -- Following supersede chains automatically (caller passes both ids). +- Following supersede chains more than one hop (omitted `new_id` resolves one + `superseded_by` link, not the full chain to the latest revision). - Diffing entities/relations/sources (claims and pages only, per ROADMAP). -- MCP/JSONL parity (`kb.*` surface unchanged). +- `ViewerContext` scope filtering on `kb.diff` (see "MCP + JSONL" above — + matches the other by-id read tools). + +Superseded decision from the original design: "MCP/JSONL parity" was +initially scoped out ("CLI-only... does not touch the `kb.*` capability +set"). Issue #327 pointed out this leaves `kb.diff` as the only read method +skipping the four-site registration convention (`CLAUDE.md` §"When you add a +new kb.* method"), so it was added — see "MCP + JSONL" above. diff --git a/docs/superpowers/specs/2026-07-07-friendlier-mcp-surface-design.md b/docs/superpowers/specs/2026-07-07-friendlier-mcp-surface-design.md new file mode 100644 index 00000000..002c7228 --- /dev/null +++ b/docs/superpowers/specs/2026-07-07-friendlier-mcp-surface-design.md @@ -0,0 +1,151 @@ +# friendlier mcp surface — tool profiles + per-prompt auto-recall — design + +date: 2026-07-07 +status: approved-direction, pending user review of this spec +decided with user: the main reason vouch feels less user-friendly than pmb is +its mcp surface — 58 tools shoved at the agent every turn. lead with slimming +that surface + auto-recall; search-fusion rides underneath as the quality layer. + +## context + +a code-grounded comparison against pmb (pmbai.dev, `oleksiijko/pmb`) — the +closest competitor — isolated *why* pmb feels smoother on first touch. it is +almost entirely the mcp surface: + +| | pmb | vouch (today) | +|---|---|---| +| tools the agent sees | **10** (default profile) | **58** (all, always) | +| gating mechanism | `PMB_TOOL_PROFILE`, minimal by default | **none** | +| tool names | intent-first (`recall`, `remember`) | mechanic-first (`kb_propose_claim`) | +| recall a memory | 0 tool calls (auto-injected each prompt) | agent must call `kb_context`/`kb_search` | +| context cost | compact one-liners, ~16kb saved | 58 verbose descriptions, paid every turn | + +pmb defines **69** tools too — it just exposes 10 by default and hides the +other 59 behind a profile flag (`_toolspec.py`, `server.py:446-478` registers +all then removes those outside the active profile). vouch surfaces its whole +propose→approve→supersede→contradict→synthesize→maintenance lifecycle at once +because the review gate is the product — but the *agent* rarely needs 50 of +those tools, and reading them every turn is the friendliness tax. + +the key insight: this is a **presentation** problem, not a capability problem. +the fix touches only which mcp tools are *shown* to a given agent. the protocol +surface, the jsonl and cli surfaces, and — critically — the review gate are all +unchanged. + +## scope + +one slice, four moves, in priority order: + +1. **mcp tool profiles** — a `minimal` default that shows ~7 core tools; the + rest move to `standard`/`full`. the biggest first-touch win, self-contained. +2. **fusion-by-default retrieval** — wire the already-built `rrf_fuse` into the + context path so recall quality is good (prerequisite for move 3 feeling + good). add recency preference + near-duplicate drop. +3. **per-prompt auto-recall** — a `UserPromptSubmit` hook that injects + relevant context every turn → recall becomes 0 tool calls, like pmb. +4. **compact tool descriptions** — one-line descriptions under non-`full` + profiles to stop burning context. nice-to-have; lands last. + +**out of scope** (later slices): the follow-rate "was it used?" loop and +reproducible A/B numbers (that's the *better-proven* slice); a warm daemon for +sub-50ms injection (only if cold-spawn latency bites); bundling a local +embedding model by default (a packaging decision with its own weight); +auto-*write* of claims from observed actions (pmb does this; for vouch it must +route through the gate as proposals — a separate design). + +## move 1 — tool profiles + +### the profiles + +- **`minimal` (default)** — the everyday knowledge loop, agent-facing: + `kb_capabilities`, `kb_context`, `kb_search`, `kb_read_page`, + `kb_propose_claim`, `kb_propose_page`, `kb_status`, `kb_list_pending`. + (8 tools.) `kb_capabilities` stays in so the agent can always discover the + wider surface and how to widen its profile. +- **`standard`** — minimal + the review lifecycle for unattended agents: + `kb_approve`, `kb_reject`, `kb_supersede`, `kb_contradict`, `kb_confirm`, + `kb_read_claim`, `kb_list_claims`, `kb_neighbors`, `kb_why`. (~16 tools.) +- **`full`** — all 58 (maintenance, provenance, eval, import/export, themes). + +approve/reject are deliberately **not** in `minimal`: approval is a human +action done at the cli or review ui, so hiding it from a first-run agent is +correct, not lossy. `standard` exists for the "let an agent run the whole loop" +case. + +### mechanism + +mirror pmb: register every `@mcp.tool()` as today, then, at server build time, +read the active profile and remove tools outside it from the fastmcp tool +manager (`mcp._tool_manager`). the profile → tool-name sets live in one place — +a new `src/vouch/mcp_profiles.py` (a dict of profile → frozenset of method +names), imported by `server.py`. config precedence: `VOUCH_TOOL_PROFILE` env +var overrides `mcp.tool_profile` in `config.yaml`, default `minimal`. + +`capabilities.METHODS` still lists all 58 — the profile filters *exposure*, not +the protocol. `mcp_profiles.py` must be exhaustive: a meta-test asserts every +name in every profile exists in `METHODS`, and that `full` == `METHODS`, so a +new tool can't silently fall out of `full`. + +## move 2 — fusion by default + +in `context.py:_retrieve`, add a `hybrid` branch that fuses semantic + fts +results via the existing `embeddings/fusion.py:rrf_fuse` instead of the current +first-non-empty waterfall; add `hybrid` to `_VALID_BACKENDS`; flip the +`storage.py` default backend to `hybrid` (gracefully degrading to fts when the +embeddings extra is absent). `auto` — what every already-initialised KB has in +its config — is redefined to mean the same fused path, so existing installs +benefit without a config migration. then, before the context-budget clip: a +greedy near-duplicate (MMR-style) drop so an agent never sees the same fact +twice. (a recency multiplier is deferred to a follow-up — it needs per-hit +timestamps that `_retrieve` does not currently carry.) the CI recall eval +(`eval/recall.py`, `eval.yml`, 0.05 regression floor) is the guardrail — +fusion should raise those numbers and can't regress them. + +## move 3 — per-prompt auto-recall + +add a `UserPromptSubmit` hook to `adapters/claude-code/.claude/settings.json` +that runs `vouch context "$PROMPT"` with a budget-capped, structured output the +harness injects as additional context. mirror into the cursor adapter and +`install_adapter.py` tiers. **approach: cold spawn first** — it runs once per +turn (not per keystroke), so a few hundred ms is usually invisible; measure it, +and only build the warm daemon (generalizing the existing openclaw rpc) if it +feels laggy. this replaces reliance on the session-start `recall` firehose, +which is not query-relevant. + +## move 4 — compact descriptions + +under `minimal`/`standard`, serve a one-line description per tool (a +`SHORT_DESC` map, or the docstring's first line) instead of the full docstring; +`full` keeps the complete docstrings. lands last; the tool-count cut is the +bulk of the context win. + +## invariants preserved + +- **review gate untouched** — no write-path change; propose stays gated; + approve/reject remain fully available (cli, review ui, `standard`/`full`). +- **git-native yaml, no saas, protocol surface unchanged** — `METHODS`, jsonl, + and cli keep all 58; only mcp *exposure* narrows. +- **3-surface parity** — the parity test is extended to enumerate the *full* + mcp tool set + cli commands against `METHODS` (closing the current 2-of-3 + gap as a bonus), checked against `full`, not the active profile. + +## verification + +- new `tests/test_mcp_profiles.py`: `minimal` exposes exactly the 8; `full` + exposes all 58; every profile name ⊆ `METHODS`; env var overrides config. +- extended `tests/test_capabilities.py`: full mcp tool set vs `METHODS` (the + real 3-surface check for the mcp surface). +- recall eval green / improved with fusion (ci-gated). +- manual before/after: fresh claude-code install shows ~8 tools not 58; + a prompt gets relevant context injected with 0 tool calls; record the + cold-spawn hook latency. + +## risks + +- **hidden tool confusion** — an agent (or user) may look for a tool that's in + `full` but not `minimal`. mitigation: `kb_capabilities` (in `minimal`) and + the docs list the profiles and how to widen with `VOUCH_TOOL_PROFILE`. +- **fusion regressing recall on some queries** — mitigated by the ci eval + floor; keep `backend` pinnable in config for escape. +- **hook latency on large KBs** — measured in verification; warm-daemon + fast-follow is the known escape hatch. diff --git a/docs/transports.md b/docs/transports.md index 64105fa3..16e16fa2 100644 --- a/docs/transports.md +++ b/docs/transports.md @@ -25,6 +25,13 @@ The server speaks MCP over stdin/stdout. Your host configures it as a subprocess. Method `kb.search` is exposed as MCP tool `kb_search` (dots aren't valid in MCP tool names). +By default only a core set of `kb_*` tools is exposed (profile +`minimal`) to keep the per-turn tool-choice cost down; set +`VOUCH_TOOL_PROFILE` (env var, wins) or `mcp.tool_profile` in +`config.yaml` to `standard` or `full` to widen the surface. Outside +`full`, each exposed tool's description is also trimmed to its first +line. + ### Resources vouch also exposes read-only views as MCP resources: diff --git a/openclaw.plugin.json b/openclaw.plugin.json index ffcd470d..381ec2bc 100644 --- a/openclaw.plugin.json +++ b/openclaw.plugin.json @@ -7,6 +7,11 @@ "skills": [ "adapters/openclaw/skills" ], + "openclaw": { + "compat": { + "pluginApi": ">=2026.6.0" + } + }, "configSchema": { "type": "object", "additionalProperties": false, diff --git a/schemas/capabilities.schema.json b/schemas/capabilities.schema.json index b986604e..f795ed3d 100644 --- a/schemas/capabilities.schema.json +++ b/schemas/capabilities.schema.json @@ -12,6 +12,12 @@ "title": "Context Engines", "type": "array" }, + "host_compat": { + "additionalProperties": true, + "description": "Per-host compatibility ranges (#237). Mirrors the `openclaw.compat` block in openclaw.plugin.json so non-OpenClaw clients can detect compat without parsing the manifest, e.g. {\"openclaw\": {\"pluginApi\": \">=2026.4.0\"}}.", + "title": "Host Compat", + "type": "object" + }, "knowledge_capability": { "additionalProperties": true, "title": "Knowledge Capability", diff --git a/src/vouch/capabilities.py b/src/vouch/capabilities.py index 3fd21c75..1393d978 100644 --- a/src/vouch/capabilities.py +++ b/src/vouch/capabilities.py @@ -7,10 +7,21 @@ from __future__ import annotations +import json +import logging +from pathlib import Path + from . import __version__ from .models import Capabilities from .openclaw.context_engine import describe_engine +_log = logging.getLogger(__name__) + +# Path to the plugin manifest, relative to this module. capabilities.py lives +# at src/vouch/capabilities.py; openclaw.plugin.json lives at the repo root, +# three levels up (src/vouch/ -> src/ -> repo root). +_PLUGIN_MANIFEST_PATH = Path(__file__).resolve().parent.parent.parent / "openclaw.plugin.json" + # The full method surface this implementation exposes. Keep this list in # sync with the MCP server + JSONL server registrations — `test_capabilities` # asserts they match. @@ -27,12 +38,14 @@ "kb.read_claim", "kb.read_entity", "kb.read_relation", + "kb.diff", "kb.list_pages", "kb.list_claims", "kb.list_entities", "kb.list_relations", "kb.list_sources", "kb.list_pending", + "kb.triage_pending", "kb.register_source", "kb.register_source_from_path", "kb.propose_claim", @@ -76,6 +89,27 @@ ] +def _load_host_compat() -> dict[str, dict[str, str]]: + """Read the `openclaw.compat` block from openclaw.plugin.json (#237). + + Surfaced in `kb.capabilities` as `host_compat` so non-OpenClaw clients + can detect compat without parsing the manifest themselves. Returns an + empty dict (rather than raising) if the manifest is missing or + malformed — capabilities() must never fail to report basic info just + because the manifest moved or this is installed as a standalone wheel + without the manifest packaged alongside it. + """ + try: + manifest = json.loads(_PLUGIN_MANIFEST_PATH.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as e: + _log.debug("openclaw.plugin.json unreadable, host_compat will be empty: %s", e) + return {} + compat = manifest.get("openclaw", {}).get("compat") + if not isinstance(compat, dict): + return {} + return {"openclaw": {k: str(v) for k, v in compat.items()}} + + def capabilities() -> Capabilities: retrieval = ["fts5", "substring"] try: @@ -98,4 +132,5 @@ def capabilities() -> Capabilities: "config_path": "retrieval.scope", }, context_engines=[describe_engine()], + host_compat=_load_host_compat(), ) diff --git a/src/vouch/cli.py b/src/vouch/cli.py index a490fcf9..c6b04ccf 100644 --- a/src/vouch/cli.py +++ b/src/vouch/cli.py @@ -25,6 +25,7 @@ from . import __version__, bundle, health, volunteer_context from . import audit as audit_mod from . import capture as capture_mod +from . import codex_rollout as codex_rollout_mod from . import compile as compile_mod from . import digest as digest_mod from . import fetch as fetch_mod @@ -98,6 +99,7 @@ def _cli_errors() -> Iterator[None]: ProposalError, LifecycleError, migrations_mod.MigrationError, + codex_rollout_mod.CodexRolloutError, ) as e: raise click.ClickException(str(e)) from e @@ -1030,6 +1032,52 @@ def list_relations() -> None: click.echo(output) +@cli.command() +@click.argument("proposal_ids", nargs=-1) +@click.option( + "--json", "as_json", is_flag=True, + help="Emit machine-readable _meta.vouch_triage blocks.", +) +@click.option( + "--reverse", is_flag=True, + help="Ascending order (worst-first) instead of the default descending (best-first).", +) +def triage(proposal_ids: tuple[str, ...], as_json: bool, reverse: bool) -> None: + """Advisory triage scoring over pending proposals (opt-in: triage.enabled). + + Scores each proposal on fit, citation quality, duplication risk, and + contradiction risk, then prints a ranked table. Never approves or + rejects — a human still decides via `vouch approve` / `vouch reject`. + """ + from . import triage as triage_mod + + store = _load_store() + with _cli_errors(): + results = triage_mod.triage_pending(store, proposal_ids=list(proposal_ids) or None) + results.sort(key=lambda r: r["_meta"]["vouch_triage"]["score"], reverse=not reverse) + + if as_json: + _emit_json(results) + return + if not results: + click.echo("no pending proposals to triage") + return + for r in results: + block = r["_meta"]["vouch_triage"] + preview = ( + r["payload"].get("text") + or r["payload"].get("title") + or r["payload"].get("name") + or r["payload"].get("id") + or "-" + ) + click.echo( + f"{block['score']:.2f} [{block['recommendation']:>11}] " + f"{r['id']} [{r['kind']}] {str(preview).strip()[:80]}" + ) + click.echo(f" {block['rationale']}") + + @cli.command() @click.argument("proposal_ids", nargs=-1, required=True) @click.option("--reason", default=None) @@ -2022,6 +2070,75 @@ def capture_finalize_all_cmd(session_id: str | None, max_age_seconds: float) -> _emit_json(result) +@capture.command("ingest-codex") +@click.argument( + "rollout", required=False, + type=click.Path(exists=True, dir_okay=False, path_type=Path), +) +@click.option( + "--latest", is_flag=True, + help="Resolve the newest codex rollout recorded for this project (by cwd).", +) +@click.option( + "--hook", "hook_mode", is_flag=True, + help="Read a codex Stop-hook payload from stdin; never fails the host " + "(exits 0 even on errors, like `capture observe`).", +) +@click.option( + "--codex-home", type=click.Path(file_okay=False, path_type=Path), default=None, + help="Codex state dir holding sessions/ (default: $CODEX_HOME or ~/.codex).", +) +def capture_ingest_codex_cmd( + rollout: Path | None, latest: bool, hook_mode: bool, codex_home: Path | None +) -> None: + """Ingest one codex session rollout into a PENDING summary proposal. + + Codex has no live notify stream vouch can use project-locally; it + persists each session as a rollout jsonl instead. This maps the + rollout's tool calls into the same observation shape `capture observe` + produces and reuses the existing rollup, so the result is the same + review-gated summary a claude session yields. Re-ingesting an + unchanged session is a no-op; a session that grew since the last + ingest refreshes its one PENDING proposal in place. Review with + `vouch review`. + """ + if hook_mode: + # Hook wire (codex `Stop` event): parse the stdin payload, resolve + # the session's rollout, ingest idempotently. Exits 0 no matter + # what — capture must never break the user's codex turn. + try: + raw = "" if sys.stdin.isatty() else sys.stdin.read() + payload = json.loads(raw) if raw.strip() else {} + if isinstance(payload, dict): + codex_rollout_mod.ingest_hook_payload( + _capture_store(), payload, codex_home=codex_home + ) + except Exception: + # the hook contract is exit 0 — never surface an error here. + pass + return + if (rollout is None) == (not latest): + raise click.ClickException("pass exactly one of ROLLOUT or --latest") + store = _load_store() + with _cli_errors(): + if latest: + found = codex_rollout_mod.find_latest_rollout( + Path.cwd(), codex_home=codex_home + ) + if found is None: + raise click.ClickException( + "no codex rollout found for this project under " + f"{(codex_home or codex_rollout_mod.default_codex_home()) / 'sessions'}; " + "pass a rollout file explicitly" + ) + rollout = found + assert rollout is not None + result = codex_rollout_mod.ingest_rollout( + store, rollout, generated_at=datetime.now(UTC).isoformat() + ) + _emit_json(result) + + @capture.command("banner") def capture_banner_cmd() -> None: """Emit a SessionStart nudge if captured summaries await review.""" @@ -2299,6 +2416,30 @@ def context( _emit_json(pack) +@cli.command(name="context-hook", hidden=True) +def context_hook() -> None: + """Emit relevant KB context for a host UserPromptSubmit hook (reads stdin). + + Wired by the claude-code adapter; not meant to be run by hand. Reads the + host's JSON hook payload on stdin, prints an additionalContext envelope, + and always exits 0 so it can never block a turn. + """ + import sys + + from . import hooks + + stdin_text = sys.stdin.read() + store = _capture_store() + out = "" + if store is not None: + try: + out = hooks.build_claude_prompt_hook(store, stdin_text) + except Exception: + out = "" + if out: + click.echo(out) + + @cli.command() @click.argument("query") @click.option("--depth", default=3, show_default=True, type=int) @@ -2883,21 +3024,33 @@ def dual_solve_cmd(issue_url: str, claude_effort: str, codex_effort: str, _emit_json({ "issue": {"number": issue.number, "title": issue.title, "url": issue.url}, + "recommendation": ds_mod.recommendation(candidates), "candidates": [ {"engine": c.engine, "branch": c.branch, "ok": c.ok, "error": c.error, "changed_files": ds_mod.changed_files(c.diff), - "diff": c.diff} for c in candidates + "log": c.log, "diff": c.diff} for c in candidates ], }) return for c in candidates: click.echo(f"\n=== {c.engine} ({c.branch}) ===", err=True) + if c.log.strip(): + click.echo("--- engine log ---", err=True) + click.echo(c.log) if c.ok: + if c.log.strip(): + click.echo("--- diff ---", err=True) click.echo(c.diff) else: click.echo(f"(failed: {c.error})", err=True) + rec = ds_mod.recommendation(candidates) + if rec.get("reason"): + label = f"recommendation: {rec['engine']}" if rec.get("engine") \ + else "recommendation: no automatic pick" + click.echo(f"{label} -- {rec['reason']}", err=True) + ok = [c for c in candidates if c.ok] if not ok: raise click.ClickException("both engines failed; nothing to choose") @@ -2979,10 +3132,14 @@ def sync_apply_cmd(source_path: str, on_conflict: str) -> None: @cli.command() @click.argument("old_id") -@click.argument("new_id") +@click.argument("new_id", required=False) @click.option("--json", "as_json", is_flag=True, default=False, help="Emit the diff as JSON.") -def diff(old_id: str, new_id: str, as_json: bool) -> None: - """Show what changed between two claim or two page revisions.""" +def diff(old_id: str, new_id: str | None, as_json: bool) -> None: + """Show what changed between two claim or two page revisions. + + NEW_ID is optional for a claim that has been superseded: it resolves to + ``superseded_by`` automatically. + """ from .diff import diff_artifacts store = _load_store() diff --git a/src/vouch/codex_rollout.py b/src/vouch/codex_rollout.py new file mode 100644 index 00000000..b23d409a --- /dev/null +++ b/src/vouch/codex_rollout.py @@ -0,0 +1,475 @@ +"""Ingest OpenAI Codex CLI session rollouts into review-gated summaries. + +Codex has no live hook stream the way claude-code does, but it persists +every session as a rollout file — ``$CODEX_HOME/sessions/YYYY/MM/DD/`` +``rollout--.jsonl`` — holding user messages, tool calls, +and outputs: everything ``capture.build_summary_body`` needs, just after +the fact instead of live. ``vouch capture ingest-codex`` maps rollout +records into the same observation shape ``capture.observe`` produces, then +reuses the existing rollup (``build_summary_body`` -> ``propose_page``) so +a codex session yields the same kind of PENDING session-summary proposal a +claude session does: one code path from observation to proposal, two front +doors. + +The rollout format is not a stable public contract. Parsing is therefore +tolerant — unknown record types are skipped — but a file that doesn't look +like a rollout at all degrades to :class:`CodexRolloutError` with an +actionable message, never a stack trace. Ingesting the same session twice +is a no-op: the session id in the rollout is the natural dedup key. + +Never calls ``approve()`` — the review gate stays intact. A human reviews +the proposal with ``vouch review`` like any other write. +""" + +from __future__ import annotations + +import json +import os +import re +from collections.abc import Iterator +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +from . import audit, capture +from .models import Proposal, ProposalStatus +from .proposals import propose_page +from .storage import KBStore + +# The default proposer when VOUCH_AGENT isn't set: rollouts are codex +# sessions, so the audit trail attributes them to the codex actor the +# adapter configures (`VOUCH_AGENT=codex` in adapters/codex/config.toml). +CODEX_ACTOR = "codex" + +_ZSTD_MAGIC = b"\x28\xb5\x2f\xfd" +_MAX_PROMPT_CHARS = 240 +_PATCH_FILE_RE = re.compile(r"^\*\*\* (Add|Update|Delete) File: (.+)$", re.MULTILINE) +_EXIT_CODE_RE = re.compile(r"exited with code (\d+)") + +# Codex tool calls that are session mechanics, not work worth summarizing. +_IGNORED_CALLS = frozenset({"update_plan", "write_stdin", "list_mcp_resources"}) +_SHELL_CALLS = frozenset({"exec_command", "shell", "local_shell", "container.exec"}) + + +class CodexRolloutError(RuntimeError): + """Raised when a rollout file can't be read or doesn't parse as one. + + The CLI layer translates this into a clean ``Error: ...`` line via + ``_cli_errors``; nothing is written to the KB when it's raised. + """ + + +@dataclass +class CodexSession: + """The capture-relevant slice of one parsed rollout.""" + + session_id: str + cwd: str | None = None + started_at: str | None = None + first_prompt: str | None = None + observations: list[dict[str, Any]] = field(default_factory=list) + + +def _clean_prompt(raw: str) -> str | None: + """Mirror ``capture.first_user_prompt``'s hygiene: skip host wrapper + messages and meta lines, collapse whitespace, cap the length.""" + text = raw.strip() + if not text or text.startswith("<"): + return None + if text.lower().startswith("caveat:"): + return None + collapsed = " ".join(text.split()) + if len(collapsed) > _MAX_PROMPT_CHARS: + collapsed = collapsed[: _MAX_PROMPT_CHARS - 1].rstrip() + "…" + return collapsed + + +def _patch_observation(patch: str) -> dict[str, Any] | None: + """Turn an apply_patch payload into an Edit/Created observation.""" + matches = _PATCH_FILE_RE.findall(patch) + if not matches: + return None + files = [path.strip() for _, path in matches] + verbs = {verb for verb, _ in matches} + if verbs == {"Add"}: + verb = "Created" + elif verbs == {"Delete"}: + verb = "Deleted" + else: + verb = "Edited" + name = files[0].rsplit("/", 1)[-1] + summary = f"{verb} {name}" if len(files) == 1 else f"{verb} {len(files)} files" + return {"tool": "Edit", "summary": summary, "files": files} + + +def _observation_from_call(name: str, arguments: object) -> dict[str, Any] | None: + """Map one codex ``function_call`` record into an observation, or None + to skip. Shapes match ``capture.summarize_tool``'s conventions so the + rollup renders codex and claude sessions identically.""" + if not name or name in _IGNORED_CALLS: + return None + args: dict[str, Any] = {} + if isinstance(arguments, str): + try: + loaded = json.loads(arguments) + if isinstance(loaded, dict): + args = loaded + except json.JSONDecodeError: + args = {} + elif isinstance(arguments, dict): + args = arguments + + if name == "apply_patch": + patch = str(args.get("input") or args.get("patch") or "") + obs = _patch_observation(patch) + if obs is not None: + return obs + + if name in _SHELL_CALLS or name == "apply_patch": + cmd = args.get("cmd") or args.get("command") or "" + if isinstance(cmd, list): + cmd = " ".join(str(c) for c in cmd) + cmd = str(cmd) + first_line = cmd.splitlines()[0] if cmd else "" + # Codex often applies patches through the shell tool as an + # `apply_patch < Iterator[str]: + """Yield the rollout's lines one at a time, decoded as UTF-8. + + Streams from the file handle rather than slurping the whole file into + memory — codex rollouts can carry large tool outputs. The zstd magic + header is checked up front (compressed rollouts aren't parsed here). + """ + try: + fh = path.open("rb") + except OSError as e: + raise CodexRolloutError(f"cannot read rollout file {path}: {e}") from e + with fh: + if fh.read(4) == _ZSTD_MAGIC: + raise CodexRolloutError( + f"{path.name} is zstd-compressed; decompress it first " + f"(`zstd -d {path.name}`) and ingest the .jsonl" + ) + fh.seek(0) + for raw_line in fh: + yield raw_line.decode("utf-8", errors="replace") + + +def parse_rollout(path: Path) -> CodexSession: + """Parse one rollout jsonl into a :class:`CodexSession`. + + Unknown record types are tolerated (the format drifts); a file that is + unreadable, compressed, or has no ``session_meta`` record raises + :class:`CodexRolloutError` with a message that says what to do next. + """ + session_id: str | None = None + cwd: str | None = None + started_at: str | None = None + first_prompt: str | None = None + observations: list[dict[str, Any]] = [] + # call_id -> index into observations, so a later function_call_output + # can mark the command as failed the way summarize_tool does live. + open_calls: dict[str, int] = {} + + for line in _iter_rollout_lines(path): + line = line.strip() + if not line: + continue + try: + record = json.loads(line) + except json.JSONDecodeError: + continue + if not isinstance(record, dict): + continue + rtype = record.get("type") + payload = record.get("payload") + if not isinstance(payload, dict): + continue + + if rtype == "session_meta" and session_id is None: + sid = payload.get("id") or payload.get("session_id") + if isinstance(sid, str) and sid.strip(): + session_id = sid.strip() + raw_cwd = payload.get("cwd") + if isinstance(raw_cwd, str) and raw_cwd: + cwd = raw_cwd + raw_ts = payload.get("timestamp") + if isinstance(raw_ts, str) and raw_ts: + started_at = raw_ts + + elif rtype == "response_item": + ptype = payload.get("type") + if ptype == "function_call": + obs = _observation_from_call( + str(payload.get("name") or ""), payload.get("arguments") + ) + if obs is not None: + observations.append(obs) + call_id = payload.get("call_id") + if isinstance(call_id, str) and obs["tool"] == "Bash": + open_calls[call_id] = len(observations) - 1 + elif ptype == "function_call_output": + call_id = payload.get("call_id") + idx = open_calls.pop(call_id, None) if isinstance(call_id, str) else None + if idx is not None: + output = payload.get("output") + match = _EXIT_CODE_RE.search(str(output)) + if match and match.group(1) != "0": + obs = observations[idx] + obs["summary"] = "Command failed: " + obs["summary"].removeprefix( + "Ran: " + ) + + elif rtype == "event_msg": + if payload.get("type") == "user_message" and first_prompt is None: + msg = payload.get("message") + if isinstance(msg, str): + first_prompt = _clean_prompt(msg) + + if session_id is None: + raise CodexRolloutError( + f"{path.name}: no session_meta record found — this doesn't look " + f"like a codex rollout, or its schema has drifted; expected a " + f"session_meta record carrying an `id`" + ) + return CodexSession( + session_id=session_id, + cwd=cwd, + started_at=started_at, + first_prompt=first_prompt, + observations=observations, + ) + + +def default_codex_home() -> Path: + env = os.environ.get("CODEX_HOME") + return Path(env) if env else Path.home() / ".codex" + + +def _rollout_meta_cwd(path: Path) -> str | None: + """The ``cwd`` recorded in a rollout's ``session_meta``, or None.""" + try: + with path.open(encoding="utf-8") as fh: + first = json.loads(fh.readline()) + except (OSError, json.JSONDecodeError, UnicodeDecodeError): + return None + if not isinstance(first, dict) or first.get("type") != "session_meta": + return None + payload = first.get("payload") + if isinstance(payload, dict) and isinstance(payload.get("cwd"), str): + return payload["cwd"] + return None + + +def find_latest_rollout(cwd: Path, *, codex_home: Path | None = None) -> Path | None: + """Newest rollout whose session ran in ``cwd``, or None. + + Rollout filenames embed the full timestamp + (``rollout-YYYY-MM-DDThh-mm-ss-.jsonl``), so ``path.name`` orders + chronologically regardless of directory. Rather than sort the whole + ``sessions/`` tree up front, keep the newest match seen and skip + opening any candidate that can't beat it — most rollouts never get read. + """ + sessions = (codex_home or default_codex_home()) / "sessions" + if not sessions.is_dir(): + return None + target = str(cwd.resolve()) + best: Path | None = None + for path in sessions.rglob("rollout-*.jsonl"): + if best is not None and path.name <= best.name: + continue + if _rollout_meta_cwd(path) == target: + best = path + return best + + +def find_existing_proposal(store: KBStore, session_id: str) -> Proposal | None: + """Any proposal (any status) already filed for this session.""" + for proposal in store.list_proposals(None): + if proposal.session_id == session_id: + return proposal + return None + + +def find_rollout_by_session_id( + session_id: str, *, codex_home: Path | None = None +) -> Path | None: + """The rollout file for one session id — codex embeds the id in the + filename (``rollout--.jsonl``), so no file needs opening. + + The session id comes from a hook payload, so it's matched as a literal + filename suffix rather than interpolated into the glob pattern: a + payload carrying glob metacharacters (``*``, ``?``, ``[``) can't widen + the search or change its semantics. + """ + sessions = (codex_home or default_codex_home()) / "sessions" + if not sessions.is_dir() or not session_id.strip(): + return None + suffix = f"-{session_id}.jsonl" + best: Path | None = None + for path in sessions.rglob("rollout-*.jsonl"): + if not path.name.endswith(suffix): + continue + if best is None or path.name > best.name: + best = path + return best + + +def _comparable_body(body: str) -> str: + """The summary body minus its generation timestamp, so re-ingesting an + unchanged rollout compares equal across runs.""" + return "\n".join( + line for line in body.splitlines() if not line.startswith("- generated:") + ) + + +def ingest_rollout( + store: KBStore, + path: Path, + *, + actor: str | None = None, + generated_at: str | None = None, +) -> dict[str, Any]: + """Roll one rollout into a PENDING summary proposal. No ``approve()``. + + Honours the same ``capture:`` config as live capture (``enabled``, + ``min_observations``) so the two front doors gate identically, and + dedups on the rollout's session id: at most one proposal per session, + ever. Because codex's Stop hook fires per *turn* rather than at session + end, re-ingesting a session that grew since the last ingest refreshes + the still-PENDING proposal in place (same id, updated summary) instead + of filing a duplicate; an unchanged rollout is a flat no-op, and a + decided proposal is history — it blocks re-ingest regardless. + """ + cfg = capture.load_config(store) + session = parse_rollout(path) + if not cfg.enabled: + return { + "session_id": session.session_id, + "captured": len(session.observations), + "summary_proposal_id": None, + "skipped": "disabled", + } + + existing = find_existing_proposal(store, session.session_id) + if existing is not None and existing.status != ProposalStatus.PENDING: + return { + "session_id": session.session_id, + "captured": len(session.observations), + "summary_proposal_id": existing.id, + "skipped": "already-ingested", + } + + if len(session.observations) < cfg.min_observations: + return { + "session_id": session.session_id, + "captured": len(session.observations), + "summary_proposal_id": None, + "skipped": "below-min", + } + + project = None + if session.cwd: + project = session.cwd.rstrip("/").rsplit("/", 1)[-1] or None + title, body = capture.build_summary_body( + session.session_id, + session.observations, + [], # no git backstop: the session's working tree is long gone + "", + project=project, + generated_at=generated_at or session.started_at, + first_prompt=session.first_prompt, + ) + resolved_actor = actor or os.environ.get("VOUCH_AGENT") or CODEX_ACTOR + + if existing is not None: + if _comparable_body(body) == _comparable_body( + str(existing.payload.get("body", "")) + ): + return { + "session_id": session.session_id, + "captured": len(session.observations), + "summary_proposal_id": existing.id, + "skipped": "already-ingested", + } + refreshed = existing.model_copy(deep=True) + refreshed.payload["title"] = title.strip() + refreshed.payload["body"] = body + store.update_proposal(refreshed) + audit.log_event( + store.kb_dir, + event="proposal.page.update", + actor=resolved_actor, + object_ids=[existing.id], + data={"reason": "codex rollout re-ingest", "captured": len(session.observations)}, + ) + return { + "session_id": session.session_id, + "captured": len(session.observations), + "summary_proposal_id": existing.id, + "updated": True, + } + + proposal = propose_page( + store, + title=title, + body=body, + page_type=capture.CAPTURE_PAGE_TYPE, + proposed_by=resolved_actor, + session_id=session.session_id, + rationale="ingested codex session rollout", + ) + return { + "session_id": session.session_id, + "captured": len(session.observations), + "summary_proposal_id": proposal.id, + } + + +def ingest_hook_payload( + store: KBStore | None, + payload: dict[str, Any], + *, + codex_home: Path | None = None, +) -> dict[str, Any] | None: + """Handle one codex Stop-hook payload; never raises. + + The hook wire (`vouch capture ingest-codex --hook`) must exit 0 even on + failure — a capture problem must never break the user's codex turn, + the same rule ``capture observe`` follows. Returns the ingest result, + or None when there was nothing safe to do. + """ + try: + if store is None: + return None + session_id = str(payload.get("session_id") or "") + if not session_id: + return None + rollout: Path | None = None + transcript = payload.get("transcript_path") + if isinstance(transcript, str) and transcript.endswith(".jsonl"): + candidate = Path(transcript) + if candidate.is_file(): + rollout = candidate + if rollout is None: + rollout = find_rollout_by_session_id(session_id, codex_home=codex_home) + if rollout is None: + return None + return ingest_rollout(store, rollout) + except Exception: + return None diff --git a/src/vouch/context.py b/src/vouch/context.py index 6e9b08ce..fa7e3f16 100644 --- a/src/vouch/context.py +++ b/src/vouch/context.py @@ -19,6 +19,7 @@ import yaml from . import graph, index_db +from .embeddings.fusion import rrf_fuse from .models import ClaimStatus, ContextItem, ContextPack, ContextQuality from .scoping import ( ViewerContext, @@ -42,7 +43,7 @@ ContextItemKind = Literal["claim", "page", "entity", "relation", "source"] -_VALID_BACKENDS = ("auto", "embedding", "fts5", "substring") +_VALID_BACKENDS = ("auto", "hybrid", "embedding", "fts5", "substring") def _configured_backend(store: KBStore) -> str: @@ -82,7 +83,8 @@ def _retrieve( """Return list of (kind, id, summary, score, backend). The backend is chosen by `retrieval.backend` in config.yaml: - - "auto" (default): embedding -> FTS5 -> substring + - "auto" (default) / "hybrid": fuse embedding + FTS5 via RRF, falling + back to a substring scan only if both retrievers are empty - "embedding": semantic search only - "fts5": lexical FTS5 only - "substring": substring scan only @@ -90,34 +92,38 @@ def _retrieve( backend = _configured_backend(store) fetch_limit = scoped_fetch_limit(limit, viewer) - if backend in ("auto", "embedding"): + if backend in ("auto", "hybrid"): + sem = index_db.search_semantic(store.kb_dir, query, limit=fetch_limit) + try: + lex = index_db.search(store.kb_dir, query, limit=fetch_limit) + except sqlite3.Error: + lex = [] + fused = rrf_fuse(sem, lex, limit=fetch_limit) + if fused: + filtered = filter_hits(store, fused, viewer, limit=limit) + return [(k, i, s, sc, "hybrid") for k, i, s, sc in filtered] + # both retrievers empty -> fall through to the substring scan below. + + if backend == "embedding": raw = index_db.search_semantic(store.kb_dir, query, limit=fetch_limit) if raw: filtered = filter_hits(store, raw, viewer, limit=limit) return [(k, i, s, sc, "embedding") for k, i, s, sc in filtered] - if backend == "embedding": - return [] + return [] - if backend in ("auto", "fts5"): + if backend == "fts5": try: hits = index_db.search(store.kb_dir, query, limit=fetch_limit) if hits: filtered = filter_hits(store, hits, viewer, limit=limit) return [(k, i, s, sc, "fts5") for k, i, s, sc in filtered] except sqlite3.Error: - # FTS5 unavailable, db missing, or schema mismatch — fall through - # to substring scan (auto) or empty (explicit fts5). Other - # exceptions are real bugs and propagate. pass - if backend == "fts5": - return [] + return [] substring_hits = store.search_substring(query, limit=fetch_limit) filtered = filter_hits(store, substring_hits, viewer, limit=limit) - return [ - (k, i, s, sc, "substring") - for k, i, s, sc in filtered - ] + return [(k, i, s, sc, "substring") for k, i, s, sc in filtered] def _enrich_summary(store: KBStore, kind: str, artifact_id: str, summary: str) -> str: @@ -197,6 +203,36 @@ def _append_graph_neighbors( return warnings +def _jaccard(a: set[str], b: set[str]) -> float: + if not a or not b: + return 0.0 + return len(a & b) / len(a | b) + + +def _dedupe_near_duplicates(items: list[ContextItem]) -> list[ContextItem]: + """Drop items whose summary is near-identical to a higher-scored one. + + The *keep* decision runs in descending-score order so the highest-scored + member of a near-duplicate cluster survives; survivors are returned in the + caller's original order. build_context_pack appends lower-priority items + (graph-expansion neighbours) after the ranked hits and relies on that tail + ordering for budget eviction, so this pass must not re-rank the pack. + + Cheap greedy heuristic (token-set Jaccard >= 0.85 over the first 40 tokens); + it can over-merge long near-templated claims that differ by a single token. + """ + dropped: set[int] = set() + kept_tokens: list[set[str]] = [] + order = sorted(range(len(items)), key=lambda i: items[i].score, reverse=True) + for idx in order: + toks = set(items[idx].summary.lower().split()[:40]) + if any(_jaccard(toks, seen) >= 0.85 for seen in kept_tokens): + dropped.add(idx) + continue + kept_tokens.append(toks) + return [it for i, it in enumerate(items) if i not in dropped] + + def build_context_pack( store: KBStore, *, @@ -254,6 +290,9 @@ def build_context_pack( rel_types=graph_rel_types, ) ) + + items = _dedupe_near_duplicates(items) + failed: list[str] = [] uncited: list[str] = [] budget_truncated = False diff --git a/src/vouch/diff.py b/src/vouch/diff.py index 8387afa0..392072e2 100644 --- a/src/vouch/diff.py +++ b/src/vouch/diff.py @@ -72,11 +72,28 @@ def _line_diff(old: str, new: str) -> list[str]: )) -def diff_artifacts(store: KBStore, old_id: str, new_id: str) -> ArtifactDiff: - """Diff two same-kind artifacts (both claims or both pages) by id.""" +def diff_artifacts(store: KBStore, old_id: str, new_id: str | None = None) -> ArtifactDiff: + """Diff two same-kind artifacts (both claims or both pages) by id. + + ``new_id`` is optional for claims: when omitted, it resolves to + ``old_id``'s ``superseded_by`` field. Pages have no successor pointer, so + omitting ``new_id`` for a page is an error. + """ old_kind = _kind_of(store, old_id) if old_kind is None: raise DiffError(f"unknown artifact: {old_id}") + + if new_id is None: + if old_kind != "claim": + raise DiffError( + f"{old_id} is a {old_kind}; pages have no successor pointer, " + "pass new_id explicitly" + ) + old_claim = store.get_claim(old_id) + if not old_claim.superseded_by: + raise DiffError(f"{old_id} has not been superseded; pass new_id explicitly") + new_id = old_claim.superseded_by + new_kind = _kind_of(store, new_id) if new_kind is None: raise DiffError(f"unknown artifact: {new_id}") diff --git a/src/vouch/dual_solve.py b/src/vouch/dual_solve.py index d7734af5..dedbb82d 100644 --- a/src/vouch/dual_solve.py +++ b/src/vouch/dual_solve.py @@ -37,6 +37,7 @@ "parse_issue_ref", "parse_summary", "prepare", + "recommendation", "record_to_kb", "repo_root", "run_candidate", @@ -73,6 +74,7 @@ class Candidate: worktree: Path diff: str = "" sha: str = "" + log: str = "" ok: bool = False error: str | None = None @@ -97,6 +99,53 @@ def changed_files(diff: str) -> list[str]: return files +def _diff_stats(candidate: Candidate) -> tuple[int, int]: + files = changed_files(candidate.diff) + return len(files), len(candidate.diff.splitlines()) + + +def recommendation(candidates: list[Candidate]) -> dict[str, str | None]: + """Return a deterministic reviewer hint for the two dual-solve candidates. + + This is deliberately a scope heuristic, not an automated quality judgment: + successful candidates beat failed ones; then the smaller changed-file count + and smaller diff win. Ties stay unresolved for the human reviewer. + """ + ok = [c for c in candidates if c.ok] + if not ok: + return { + "engine": None, + "reason": "neither engine produced a usable diff.", + } + if len(ok) == 1: + return { + "engine": ok[0].engine, + "reason": f"only {ok[0].engine} produced a usable diff.", + } + + ranked = sorted(ok, key=_diff_stats) + best = ranked[0] + other = ranked[1] + best_files, best_lines = _diff_stats(best) + other_files, other_lines = _diff_stats(other) + if (best_files, best_lines) == (other_files, other_lines): + return { + "engine": None, + "reason": ( + "both engines produced equally scoped diffs; " + "review the logs and tests before choosing." + ), + } + return { + "engine": best.engine, + "reason": ( + f"{best.engine} has the smaller scoped diff " + f"({best_files} files, {best_lines} lines vs " + f"{other_files} files, {other_lines} lines)." + ), + } + + def parse_issue_ref(ref: str) -> tuple[str | None, str]: """Normalize an issue reference for ``gh issue view``. @@ -209,7 +258,7 @@ def run_candidate(engine: Engine, issue: Issue, prompt: str, root: Path, return cand try: - engine.fix(cwd=str(worktree), prompt=prompt) + cand.log = engine.fix(cwd=str(worktree), prompt=prompt) except Exception as exc: cand.error = f"engine failed: {exc}" return cand diff --git a/src/vouch/hooks.py b/src/vouch/hooks.py new file mode 100644 index 00000000..e90c2f66 --- /dev/null +++ b/src/vouch/hooks.py @@ -0,0 +1,68 @@ +"""Host-hook helpers: translate an agent host's prompt hook into KB context. + +Claude Code's UserPromptSubmit hook passes a JSON payload on stdin and injects +whatever the hook prints (as an `additionalContext` envelope) before the model +runs. `build_claude_prompt_hook` turns that payload into a compact, relevant +context block drawn from *approved* KB knowledge — so recall costs the agent +zero tool calls. It never raises: on any problem it returns "" (inject +nothing), so a hook failure can never block the user's turn. +""" + +from __future__ import annotations + +import json +import logging +from typing import Any + +from .context import build_context_pack +from .storage import KBStore + +_log = logging.getLogger("vouch") + +_MAX_ITEMS = 8 +_MAX_CHARS = 2000 + + +def _render(pack: dict[str, Any]) -> str: + lines: list[str] = [] + for item in pack.get("items", []): + summary = str(item.get("summary", "")).strip() + if not summary: + continue + cites = item.get("citations") or [] + suffix = f" [{', '.join(cites)}]" if cites else "" + lines.append(f"- {summary}{suffix}") + return "\n".join(lines) + + +def build_claude_prompt_hook(store: KBStore, stdin_text: str) -> str: + """Return the stdout a host should inject for this prompt, or "" for none.""" + try: + payload = json.loads(stdin_text) if stdin_text.strip() else {} + except json.JSONDecodeError: + payload = {"prompt": stdin_text} + if not isinstance(payload, dict): + payload = {} + prompt = str(payload.get("prompt", "")).strip() + if not prompt: + return "" + try: + pack = build_context_pack( + store, query=prompt, limit=_MAX_ITEMS, max_chars=_MAX_CHARS, + ) + except Exception: + _log.warning("context-hook: build_context_pack failed", exc_info=True) + return "" + body = _render(pack) if isinstance(pack, dict) else "" + if not body: + return "" + block = ( + "Relevant knowledge from the project's vouch KB " + "(approved & cited — consider it before answering):\n" + body + ) + return json.dumps({ + "hookSpecificOutput": { + "hookEventName": "UserPromptSubmit", + "additionalContext": block, + } + }) diff --git a/src/vouch/install_adapter.py b/src/vouch/install_adapter.py index ba4f0b1c..4fce5cd0 100644 --- a/src/vouch/install_adapter.py +++ b/src/vouch/install_adapter.py @@ -12,6 +12,9 @@ inside a `` ... `` block (``InstallResult.appended``). If the fence already exists, the file is treated as skipped -- so reruns of ``vouch install-mcp`` stay flat-noop. +* **settings.json with ``json_merge`` / config.toml with ``toml_merge``** -> + an existing destination is deep-merged into instead of skipped + (``InstallResult.merged``); the user's existing values always win. Tiers stack from T1 (the minimum: MCP wire) through T4 (full integration: slash commands and host-side hooks). Each manifest declares only the tiers @@ -27,8 +30,11 @@ from __future__ import annotations +import datetime import json +import re import shutil +import tomllib from dataclasses import dataclass, field from pathlib import Path from typing import Any @@ -75,6 +81,7 @@ class _FileEntry: dst: str # path relative to the target directory fenced_append: bool = False # CLAUDE.md-style: append inside our fence json_merge: bool = False # settings.json-style: deep-merge into existing + toml_merge: bool = False # config.toml-style: deep-merge into existing @dataclass(frozen=True) @@ -135,10 +142,32 @@ def _load_manifest(host: str) -> _Manifest: raise AdapterError( f"{host}: install.yaml tier {tier_name}: every entry needs a non-empty `dst`" ) - fenced = bool(raw.get("fenced_append", False)) - json_merge = bool(raw.get("json_merge", False)) + def _flag(name: str, raw: Any = raw, tier_name: str = tier_name) -> bool: + # Require an actual YAML boolean. `bool(raw.get(...))` would + # coerce a mistakenly-quoted `toml_merge: "false"` (a + # non-empty string) to True and silently enable a merge + # strategy, so reject anything that isn't a real bool. + val = raw.get(name, False) + if not isinstance(val, bool): + raise AdapterError( + f"{host}: install.yaml tier {tier_name}: `{name}` must be " + f"a boolean, got {type(val).__name__} ({val!r})" + ) + return val + + fenced = _flag("fenced_append") + json_merge = _flag("json_merge") + toml_merge = _flag("toml_merge") + if fenced + json_merge + toml_merge > 1: + raise AdapterError( + f"{host}: install.yaml tier {tier_name}: entry sets more than " + f"one of fenced_append/json_merge/toml_merge; pick one strategy" + ) parsed_entries.append( - _FileEntry(src=src, dst=dst, fenced_append=fenced, json_merge=json_merge) + _FileEntry( + src=src, dst=dst, fenced_append=fenced, + json_merge=json_merge, toml_merge=toml_merge, + ) ) if parsed_entries: parsed[tier_name] = parsed_entries @@ -224,6 +253,10 @@ def install(adapter: str, *, target: Path, tier: str = "T4") -> InstallResult: _install_json_merge(src, dst, result, entry.dst) continue + if entry.toml_merge: + _install_toml_merge(src, dst, result, entry.dst) + continue + if dst.exists(): result.skipped.append(entry.dst) continue @@ -247,10 +280,15 @@ def _install_fenced( States: - * dst is missing -> write fresh, fenced (``written``) - * dst exists, fence not in file -> append fenced block (``appended``) - * dst exists, fence already in -> skip (``skipped``); we are the - author and there's nothing to do + * dst is missing -> write fresh, fenced (``written``) + * dst exists, fence not in file -> append fenced block (``appended``) + * dst exists, fence body up to date -> skip (``skipped``); we are the + author and there's nothing to do + * dst exists, fence body edited -> replace within the markers + (``merged``); the fence is ours, + content around it is the user's + * dst exists, begin without end -> skip (``skipped``); corrupt fence + we refuse to mangle """ snippet = src.read_text(encoding="utf-8") fenced_block = f"\n{manifest.fence_begin}\n{snippet.rstrip()}\n{manifest.fence_end}\n" @@ -262,16 +300,71 @@ def _install_fenced( return existing = dst.read_text(encoding="utf-8") - if manifest.fence_begin in existing: + span = _fence_span(existing, manifest.fence_begin, manifest.fence_end) + if span is not None: + start, stop = span + current = existing[start:stop] + expected = fenced_block.strip("\n") + if current == expected: + result.skipped.append(rel_dst) + return + # The fence is ours: bring an edited body back in sync in place, + # touching nothing outside the markers. + refreshed = existing[:start] + expected + existing[stop:] + dst.write_text(refreshed, encoding="utf-8") + result.merged.append(rel_dst) + return + + if _has_standalone_line(existing, manifest.fence_begin): + # A begin marker on its own line with no matching end marker: a + # corrupt fence we can't cleanly parse. Don't append a second one. result.skipped.append(rel_dst) return # User-authored content above; append our fenced block at the bottom. + # (A file that merely *mentions* the marker text in prose or a code + # sample has no standalone fence, so it lands here and gets the block + # appended rather than being mistaken for an existing install.) new_content = existing.rstrip() + "\n" + fenced_block dst.write_text(new_content, encoding="utf-8") result.appended.append(rel_dst) +def _has_standalone_line(text: str, marker: str) -> bool: + return any(line.strip() == marker for line in text.splitlines()) + + +def _fence_span(text: str, begin: str, end: str) -> tuple[int, int] | None: + """Char offsets of a well-formed fence whose ``begin``/``end`` markers + each occupy their own line, or None if there's no such pair. + + Only standalone marker lines count — text that merely mentions the + marker inside prose or a fenced code sample is ignored, so a passing + reference can't be mistaken for an installed fence (and can't cause an + in-place rewrite to clobber unrelated content between two stray + mentions). The returned span runs from the start of the begin line to + the end of the end-marker text (excluding its trailing newline), so a + caller can splice a replacement in without disturbing the surrounding + file. + """ + lines = text.splitlines(keepends=True) + begin_idx: int | None = None + end_idx: int | None = None + for i, line in enumerate(lines): + stripped = line.strip() + if begin_idx is None: + if stripped == begin: + begin_idx = i + elif stripped == end: + end_idx = i + break + if begin_idx is None or end_idx is None: + return None + start = sum(len(line) for line in lines[:begin_idx]) + stop = sum(len(line) for line in lines[:end_idx]) + len(lines[end_idx].rstrip("\n")) + return start, stop + + def _event_commands(groups: Any) -> set[str]: """Every hook ``command`` string already present under one hooks-event.""" cmds: set[str] = set() @@ -389,3 +482,144 @@ def _install_json_merge( result.merged.append(rel_dst) else: result.skipped.append(rel_dst) + + +def _merge_toml(src: dict[str, Any], dst: dict[str, Any]) -> bool: + """Recursively add ``src`` keys missing from ``dst`` in place. Returns + True if ``dst`` changed. Same never-clobber convention as + :func:`_merge_settings`: on any conflict — a key present on both sides + with non-table values, or with mismatched types — the user's existing + ``dst`` value wins and only genuinely missing nested keys are filled in. + Idempotent: re-merging the same ``src`` reports no change. + """ + changed = False + for key, src_val in src.items(): + if key not in dst: + dst[key] = src_val + changed = True + continue + dst_val = dst[key] + if ( + isinstance(src_val, dict) + and isinstance(dst_val, dict) + and _merge_toml(src_val, dst_val) + ): + changed = True + return changed + + +_BARE_TOML_KEY = re.compile(r"[A-Za-z0-9_-]+") + + +def _toml_key(key: str) -> str: + if _BARE_TOML_KEY.fullmatch(key): + return key + # TOML basic strings share JSON's escape rules, so json.dumps is a + # valid quoted-key serializer. + return json.dumps(key) + + +def _toml_inline(value: Any) -> str: + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, int): + return str(value) + if isinstance(value, float): + if value != value or value in (float("inf"), float("-inf")): + raise ValueError(f"non-finite float not serialized: {value!r}") + return repr(value) + if isinstance(value, str): + return json.dumps(value) + if isinstance(value, (datetime.datetime, datetime.date, datetime.time)): + return value.isoformat() + if isinstance(value, list): + return "[" + ", ".join(_toml_inline(v) for v in value) + "]" + if isinstance(value, dict): + pairs = ", ".join( + f"{_toml_key(str(k))} = {_toml_inline(v)}" for k, v in value.items() + ) + return "{" + pairs + "}" + raise ValueError(f"unsupported TOML value type: {type(value).__name__}") + + +def _emit_toml_table( + table: dict[str, Any], path: list[str], lines: list[str] +) -> None: + plain = [(k, v) for k, v in table.items() if not isinstance(v, dict)] + subs = [(k, v) for k, v in table.items() if isinstance(v, dict)] + # A header is only needed for the table's own keys, or to make an empty + # table exist at all; sub-table headers imply their parents. + if path and (plain or not subs): + if lines: + lines.append("") + lines.append("[" + ".".join(_toml_key(p) for p in path) + "]") + for key, value in plain: + lines.append(f"{_toml_key(str(key))} = {_toml_inline(value)}") + for key, value in subs: + _emit_toml_table(value, [*path, str(key)], lines) + + +def _toml_dumps(data: dict[str, Any]) -> str: + """Serialize the merged config back to TOML. + + Deliberately minimal — covers the shapes tomllib can produce from the + configs we merge into (tables, arrays, inline tables inside arrays, + scalars, datetimes), not the whole spec. Lists containing tables are + emitted as arrays of inline tables rather than ``[[table]]`` blocks. + Raises ValueError on anything it can't faithfully re-emit; the caller + treats that as "leave the user's file alone". + """ + lines: list[str] = [] + _emit_toml_table(data, [], lines) + return "\n".join(lines) + "\n" if lines else "" + + +def _install_toml_merge( + src: Path, dst: Path, result: InstallResult, rel_dst: str +) -> None: + """config.toml-style: deep-merge our tables into a pre-existing TOML + file instead of skipping it. ``.codex/config.toml`` is codex's primary + config file, so a plain copy-or-skip would leave vouch unwired on any + project where codex is already configured (vouchdev/vouch#384). + + States mirror :func:`_install_json_merge`: + + * dst missing -> copy fresh (``written``) + * dst exists, merge adds keys -> merge + rewrite (``merged``) + * dst exists, nothing to add -> skip (``skipped``); already installed + * dst exists, unparseable -> skip (``skipped``); never clobber the user + + Rewriting re-serializes the whole file (comments and formatting are not + preserved — same trade-off ``_install_json_merge`` already makes). The + serialized result must survive a tomllib round-trip back to the merged + data; anything the minimal serializer can't faithfully re-emit degrades + to ``skipped`` rather than risking the user's config. + """ + if not dst.exists(): + dst.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(src, dst) + result.written.append(rel_dst) + return + + try: + dst_data = tomllib.loads(dst.read_text(encoding="utf-8")) + src_data = tomllib.loads(src.read_text(encoding="utf-8")) + except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError): + # Malformed or unreadable user file — leave it untouched. + result.skipped.append(rel_dst) + return + + if not _merge_toml(src_data, dst_data): + result.skipped.append(rel_dst) + return + + try: + text = _toml_dumps(dst_data) + if tomllib.loads(text) != dst_data: + raise ValueError("serializer round-trip mismatch") + except (ValueError, tomllib.TOMLDecodeError): + result.skipped.append(rel_dst) + return + + dst.write_text(text, encoding="utf-8") + result.merged.append(rel_dst) diff --git a/src/vouch/jsonl_server.py b/src/vouch/jsonl_server.py index 5e3b62b7..3ada4b9a 100644 --- a/src/vouch/jsonl_server.py +++ b/src/vouch/jsonl_server.py @@ -18,9 +18,9 @@ from __future__ import annotations import json +import logging import os import sys -import traceback from collections.abc import Callable from contextvars import ContextVar from pathlib import Path @@ -63,6 +63,8 @@ ) from .synthesize import synthesize +_log = logging.getLogger("vouch.jsonl_server") + # Per-request actor override. The HTTP transport sets this from the # X-Vouch-Agent header so audit attribution is correct without mutating # process-wide env (each ThreadingHTTPServer request thread gets its own @@ -249,6 +251,14 @@ def _h_read_relation(p: dict) -> dict: return _store().get_relation(p["relation_id"]).model_dump(mode="json") +def _h_diff(p: dict) -> dict: + from dataclasses import asdict + + from .diff import diff_artifacts + + return asdict(diff_artifacts(_store(), p["old_id"], p.get("new_id"))) + + def _h_list_pages(p: dict) -> list[dict]: pages = filter_pages( _store().list_pages(), @@ -294,6 +304,12 @@ def _h_list_pending(_: dict) -> list[dict]: ] +def _h_triage_pending(p: dict) -> list[dict]: + from . import triage as triage_mod + + return triage_mod.triage_pending(_store(), proposal_ids=p.get("proposal_ids")) + + def _h_register_source(p: dict) -> dict: s = _store() src = s.put_source( @@ -723,12 +739,14 @@ def _h_propose_theme(p: dict) -> dict: "kb.read_claim": _h_read_claim, "kb.read_entity": _h_read_entity, "kb.read_relation": _h_read_relation, + "kb.diff": _h_diff, "kb.list_pages": _h_list_pages, "kb.list_claims": _h_list_claims, "kb.list_entities": _h_list_entities, "kb.list_relations": _h_list_relations, "kb.list_sources": _h_list_sources, "kb.list_pending": _h_list_pending, + "kb.triage_pending": _h_triage_pending, "kb.register_source": _h_register_source, "kb.register_source_from_path": _h_register_source_from_path, "kb.propose_claim": _h_propose_claim, @@ -800,12 +818,12 @@ def handle_request(envelope: dict) -> dict: "error": {"code": "invalid_request", "message": str(e)}, } except Exception as e: + _log.exception("internal error handling %s", method) return { "id": req_id, "ok": False, "error": { "code": "internal_error", "message": str(e), - "traceback": traceback.format_exc(), }, } diff --git a/src/vouch/mcp_profiles.py b/src/vouch/mcp_profiles.py new file mode 100644 index 00000000..6f79f7c3 --- /dev/null +++ b/src/vouch/mcp_profiles.py @@ -0,0 +1,109 @@ +"""MCP tool profiles — narrow the tool surface an agent sees by default. + +vouch exposes 58 kb.* methods. Handing all of them to an agent every turn is +the main first-touch friendliness cost (the closest competitor, pmb, exposes +~10 by default and hides the rest behind a profile flag). Profiles control +*exposure* only: the JSONL and CLI surfaces, the protocol method list +(capabilities.METHODS), and the review gate are unchanged. + +Resolution order: VOUCH_TOOL_PROFILE env var > config.yaml `mcp.tool_profile` +> "minimal" (the default). Unknown names fall back to "minimal". +""" + +from __future__ import annotations + +import os +from typing import Any + +from .capabilities import METHODS + +_MINIMAL: frozenset[str] = frozenset({ + "kb.capabilities", + "kb.status", + "kb.context", + "kb.search", + "kb.read_page", + "kb.propose_claim", + "kb.propose_page", + "kb.list_pending", +}) + +_STANDARD: frozenset[str] = _MINIMAL | frozenset({ + "kb.approve", + "kb.reject", + "kb.supersede", + "kb.contradict", + "kb.confirm", + "kb.read_claim", + "kb.list_claims", + "kb.neighbors", + "kb.why", +}) + +PROFILES: dict[str, frozenset[str]] = { + "minimal": _MINIMAL, + "standard": _STANDARD, + "full": frozenset(METHODS), +} + +DEFAULT_PROFILE = "minimal" + + +def _tool_name(method: str) -> str: + """`"kb.propose_claim"` -> `"kb_propose_claim"` (MCP tool names use `_`).""" + return "kb_" + method.split(".", 1)[1] + + +def resolve_profile_name(config: dict[str, Any] | None = None) -> str: + """Pick the active profile from env > config > default.""" + raw = os.environ.get("VOUCH_TOOL_PROFILE") + if not raw and isinstance(config, dict): + mcp_cfg = config.get("mcp") + if isinstance(mcp_cfg, dict): + raw = mcp_cfg.get("tool_profile") + name = str(raw).strip().lower() if raw else DEFAULT_PROFILE + return name if name in PROFILES else DEFAULT_PROFILE + + +def tool_names_for(name: str) -> set[str]: + """The MCP (underscore) tool names exposed by profile `name`.""" + return {_tool_name(m) for m in PROFILES.get(name, PROFILES[DEFAULT_PROFILE])} + + +def apply_tool_profile(mcp: Any, name: str) -> list[str]: + """Remove every registered `kb_*` MCP tool not in profile `name`. + + Returns the sorted list of removed tool names. Idempotent. `full` removes + nothing. Only `kb_*` tools are touched, so trust/diagnostic tools + registered elsewhere are never dropped. + """ + keep = tool_names_for(name) + tools = mcp._tool_manager._tools + removed: list[str] = [] + for tool_name in list(tools.keys()): + if tool_name.startswith("kb_") and tool_name not in keep: + tools.pop(tool_name, None) + removed.append(tool_name) + return sorted(removed) + + +def compact_descriptions(mcp: Any) -> int: + """Trim each kb_ tool's description to its first line to save context. + + Full docstrings are only needed under the `full` profile; the first line + is enough for an agent choosing a tool. Returns the number changed. + """ + changed = 0 + for tool in mcp._tool_manager._tools.values(): + if not tool.name.startswith("kb_"): + continue + desc = tool.description or "" + first = desc.strip().split("\n", 1)[0].strip() + if first and first != desc: + try: + tool.description = first + except (AttributeError, TypeError): + # pydantic frozen-model fallback + object.__setattr__(tool, "description", first) + changed += 1 + return changed diff --git a/src/vouch/models.py b/src/vouch/models.py index dccf4453..5159d8ff 100644 --- a/src/vouch/models.py +++ b/src/vouch/models.py @@ -82,6 +82,18 @@ def _coerce_artifact_scope(value: object) -> object: return value +def _require_non_empty(v: str, label: str) -> str: + """Reject empty / whitespace-only required text fields (#155). + + Shared by the ``Claim.text`` / ``Entity.name`` / ``Page.title`` + validators. Gates on emptiness only — non-blank values pass through + unchanged, so surrounding whitespace is preserved. + """ + if not v.strip(): + raise ValueError(f"{label} must not be empty") + return v + + class ArtifactScope(BaseModel): """Structured scope: visibility tier plus optional project/agent binding.""" @@ -232,6 +244,16 @@ def _at_least_one_citation(cls, v: list[str]) -> list[str]: "(README §'Object model'; CONTRIBUTING §'Things we won't merge')" ) return v + + @field_validator("text") + @classmethod + def _text_non_empty(cls, v: str) -> str: + # Same shape as _at_least_one_citation: the non-empty contract lived + # only in proposals.propose_claim, so store.put_claim, + # store.update_claim, and bundle.import_apply via _validate_content + # accepted text="" / whitespace and landed a claim carrying zero + # semantic content. Enforce on the model to close all paths at once. + return _require_non_empty(v, "claim text") entities: list[str] = Field(default_factory=list) supersedes: list[str] = Field(default_factory=list) superseded_by: str | None = None @@ -267,6 +289,13 @@ class Entity(BaseModel): created_at: datetime = Field(default_factory=utcnow) updated_at: datetime = Field(default_factory=utcnow) + @field_validator("name") + @classmethod + def _name_non_empty(cls, v: str) -> str: + # See Claim._text_non_empty — the propose_entity check alone left + # store.put_entity and bundle import accepting name="" / whitespace. + return _require_non_empty(v, "entity name") + class Relation(BaseModel): """Typed edge between entities / claims / pages.""" @@ -315,6 +344,13 @@ def _normalize_type(cls, v: Any) -> str: raise ValueError("page type must be a non-empty string") return v.strip() + @field_validator("title") + @classmethod + def _title_non_empty(cls, v: str) -> str: + # See Claim._text_non_empty — the propose_page check alone left + # store.put_page and bundle import accepting title="" / whitespace. + return _require_non_empty(v, "page title") + # --- audit + sessions ----------------------------------------------------- @@ -457,3 +493,12 @@ class Capabilities(BaseModel): default_factory=list, description="OpenClaw context engines exposed (see openclaw.plugin.json)", ) + host_compat: dict[str, Any] = Field( + default_factory=dict, + description=( + "Per-host compatibility ranges (#237). Mirrors the " + "`openclaw.compat` block in openclaw.plugin.json so non-OpenClaw " + "clients can detect compat without parsing the manifest, e.g. " + '{"openclaw": {"pluginApi": ">=2026.4.0"}}.' + ), + ) diff --git a/src/vouch/server.py b/src/vouch/server.py index 8cabee08..f2a4482e 100644 --- a/src/vouch/server.py +++ b/src/vouch/server.py @@ -19,7 +19,7 @@ import yaml from mcp.server.fastmcp import FastMCP -from . import audit, bundle, health, volunteer_context +from . import audit, bundle, health, mcp_profiles, volunteer_context from . import compile as compile_mod from . import digest as digest_mod from . import lifecycle as life @@ -309,6 +309,18 @@ def kb_read_relation(relation_id: str) -> dict[str, Any]: raise ValueError(str(e)) from e +@mcp.tool() +def kb_diff(old_id: str, new_id: str | None = None) -> dict[str, Any]: + """Field-level diff between two claim revisions or two page revisions. + + new_id is optional for a superseded claim: resolves to superseded_by. + """ + from dataclasses import asdict + + from .diff import diff_artifacts + return asdict(diff_artifacts(_store(), old_id, new_id)) + + @mcp.tool() def kb_list_pages( *, @@ -382,6 +394,23 @@ def kb_list_pending() -> list[dict[str, Any]]: ] +@mcp.tool() +def kb_triage_pending(proposal_ids: list[str] | None = None) -> list[dict[str, Any]]: + """Advisory triage scoring over the pending-review queue. + + Attaches `_meta.vouch_triage` (recommendation/score/signals/rationale) + to each pending proposal's view. Read-only — never approves, rejects, + or otherwise decides; a human still calls `kb_approve` / `kb_reject`. + Opt-in: disabled unless `triage.enabled: true` is set in config.yaml. + """ + from . import triage as triage_mod + + try: + return triage_mod.triage_pending(_store(), proposal_ids=proposal_ids) + except (ValueError, ArtifactNotFoundError) as e: + raise ValueError(str(e)) from e + + # === write tools — gated (produce proposals) ============================= @@ -1012,4 +1041,12 @@ def run_stdio() -> None: """Entry point used by `vouch serve`.""" configure_logging() trust_mod.set_stdio_default(trust_mod.MCP_STDIO) + try: + cfg: dict[str, Any] | None = _load_cfg(_store()) + except Exception: + cfg = None + profile = mcp_profiles.resolve_profile_name(cfg) + mcp_profiles.apply_tool_profile(mcp, profile) + if profile != "full": + mcp_profiles.compact_descriptions(mcp) mcp.run() diff --git a/src/vouch/sessions.py b/src/vouch/sessions.py index 44e8b285..d243646b 100644 --- a/src/vouch/sessions.py +++ b/src/vouch/sessions.py @@ -14,9 +14,9 @@ from . import audit, index_db, volunteer_context from . import salience as salience_mod -from .models import Page, PageType, ProposalStatus, Session +from .models import Page, PageType, ProposalKind, ProposalStatus, Session from .proposals import approve -from .storage import KBStore +from .storage import ArtifactNotFoundError, KBStore logger = logging.getLogger(__name__) @@ -99,18 +99,24 @@ def crystallize( }) summary_page_id: str | None = None - if write_summary_page and approved_artifact_ids: + approved_for_session = _approved_artifact_ids_for_session(store, sess.id) + if write_summary_page and approved_for_session: page = Page( id=f"session-{sess.id}", title=f"Session {sess.id}", type=PageType.SESSION, - body=_build_summary_body(sess, approved_artifact_ids), + body=_build_summary_body(sess, approved_for_session), claims=[ - aid for aid in approved_artifact_ids + aid for aid in approved_for_session if (store.kb_dir / "claims" / f"{aid}.yaml").exists() ], ) - store.put_page(page) + try: + store.get_page(page.id) + except ArtifactNotFoundError: + store.put_page(page) + else: + store.update_page(page) with index_db.open_db(store.kb_dir) as conn: index_db.index_page( conn, id=page.id, title=page.title, body=page.body, @@ -155,3 +161,14 @@ def _build_summary_body(sess: Session, ids: list[str]) -> str: for aid in ids: lines.append(f"- `{aid}`") return "\n".join(lines) + + +def _approved_artifact_ids_for_session(store: KBStore, session_id: str) -> list[str]: + ids = { + str(pr.payload.get("id")) + for pr in store.list_proposals(ProposalStatus.APPROVED) + if pr.session_id == session_id and pr.kind in { + ProposalKind.CLAIM, ProposalKind.PAGE, ProposalKind.ENTITY, ProposalKind.RELATION, + } and pr.payload.get("id") + } + return sorted(ids) diff --git a/src/vouch/storage.py b/src/vouch/storage.py index 653ea806..3a057db3 100644 --- a/src/vouch/storage.py +++ b/src/vouch/storage.py @@ -91,9 +91,9 @@ def _starter_config() -> dict[str, Any]: "max_chars": 12000, }, "retrieval": { - # auto = embedding -> fts5 -> substring; or pin one of + # hybrid/auto = fuse embedding + fts5 via RRF; or pin one of # embedding | fts5 | substring. See context._retrieve. - "backend": "auto", + "backend": "hybrid", "default_limit": 10, }, "agents": { @@ -815,6 +815,20 @@ def put_proposal(self, proposal: Proposal) -> Proposal: ) from e return proposal + def update_proposal(self, proposal: Proposal) -> Proposal: + """Overwrite an existing *pending* proposal file in place. + + Pure I/O: the caller decides whether a refresh is legitimate (only + pending proposals may be rewritten — a decided one is history). + """ + path = self._proposal_path(proposal.id) + if not path.exists(): + raise ArtifactNotFoundError(f"proposal {proposal.id}") + path.write_text( + _yaml_dump(proposal.model_dump(mode="json")), encoding="utf-8" + ) + return proposal + def get_proposal(self, proposal_id: str) -> Proposal: for path in (self._proposal_path(proposal_id), self._decided_path(proposal_id)): if path.exists(): diff --git a/src/vouch/triage.py b/src/vouch/triage.py new file mode 100644 index 00000000..31202e40 --- /dev/null +++ b/src/vouch/triage.py @@ -0,0 +1,503 @@ +"""Advisory triage scoring for the pending-review queue (issue #322). + +Read-only. Scores each pending proposal on four signals — fit, citation +quality, duplication risk, and contradiction risk — and folds them into a +composite ``score`` plus an advisory ``recommendation``. The result is +attached as ``_meta.vouch_triage`` on the proposal's own ``model_dump``. + +This never decides anything: no call here ever reaches +``proposals.approve``, ``proposals.reject``, ``store.put_*``, or +``store.move_proposal_to_decided``. A human still calls ``kb.approve`` / +``kb.reject``; ``recommendation`` is a hint the reviewer may ignore. + +Opt-in: disabled unless ``triage.enabled: true`` is set in +``.vouch/config.yaml`` (mirrors the defensive yaml-read pattern in +``salience.reflex_cfg`` / ``embeddings.similarity.similarity_threshold`` — +no pydantic Config model yet, see issue #243). + +Duplication risk reuses the embedding path already built for propose-time +warnings (``embeddings.similarity.find_similar_on_propose``); fit uses the +same underlying primitive (``index_db.search_embedding``) at a lower +threshold band so a near-duplicate hit doesn't also inflate fit and cancel +out its own duplication penalty (see ``_topical_fit_scores``). When no +embedder is registered (base install, no ``[embeddings]`` extra), both +signals fall back to a ``difflib`` text-similarity heuristic so the method +still returns a full block. +""" + +from __future__ import annotations + +import difflib +import re +from dataclasses import dataclass +from typing import Any + +import yaml + +from .models import Proposal, ProposalKind, ProposalStatus +from .proposals import _payload_block_reason +from .storage import ArtifactNotFoundError, KBStore + +DEFAULT_WEIGHTS: dict[str, float] = { + "fit": 0.3, + "citation_quality": 0.3, + "duplication_risk": 0.2, + "contradiction_risk": 0.2, +} + +_APPROVE_THRESHOLD = 0.7 +_REJECT_THRESHOLD = 0.35 +_FUZZY_MATCH_FLOOR = 0.3 +_CONTRADICTION_CANDIDATE_FLOOR = 0.35 + +_NEGATION_MARKERS = frozenset({ + "not", "no", "never", "cannot", "isnt", "doesnt", "wont", "wasnt", + "arent", "dont", "didnt", "hasnt", "havent", "without", "neither", "nor", +}) + + +class TriageError(ValueError): + """Raised when `kb.triage_pending` is invoked while disabled, or misused.""" + + +@dataclass(frozen=True) +class TriageConfig: + enabled: bool + backend: str + weights: dict[str, float] + + +def triage_cfg(store: KBStore) -> TriageConfig: + """Read `triage.*` from config.yaml defensively. Default: disabled.""" + cfg: dict[str, Any] = {} + try: + loaded = yaml.safe_load(store.config_path.read_text(encoding="utf-8")) + if isinstance(loaded, dict): + cfg = loaded + except Exception: + pass + + triage = cfg.get("triage") + triage = triage if isinstance(triage, dict) else {} + + enabled = triage.get("enabled", False) + enabled = bool(enabled) if isinstance(enabled, bool) else False + + backend = triage.get("backend", "embeddings") + backend = backend if isinstance(backend, str) else "embeddings" + + weights = dict(DEFAULT_WEIGHTS) + weights_cfg = triage.get("weights") + if isinstance(weights_cfg, dict): + for key in weights: + value = weights_cfg.get(key) + if isinstance(value, int | float) and not isinstance(value, bool): + weights[key] = float(value) + + return TriageConfig(enabled=enabled, backend=backend, weights=weights) + + +# --- shared helpers --------------------------------------------------------- + + +def _referenced_entity_ids(proposal: Proposal) -> list[str]: + if proposal.kind in (ProposalKind.CLAIM, ProposalKind.PAGE): + return list(proposal.payload.get("entities") or []) + return [] + + +def _has_negation(text: str) -> bool: + tokens = set(re.findall(r"[a-z']+", text.casefold())) + tokens = {t.replace("'", "") for t in tokens} + return bool(tokens & _NEGATION_MARKERS) + + +def _safe_embedder() -> Any | None: + try: + from .embeddings import get_embedder + + return get_embedder() + except Exception: + return None + + +def _best_fuzzy_match( + text: str, pool: list[tuple[str, str]], +) -> tuple[str | None, float]: + needle = text.casefold() + best_id: str | None = None + best_ratio = 0.0 + for cid, candidate in pool: + candidate = (candidate or "").strip() + if not candidate: + continue + ratio = difflib.SequenceMatcher(None, needle, candidate.casefold()).ratio() + if ratio > best_ratio: + best_ratio, best_id = ratio, cid + if best_id is None or best_ratio < _FUZZY_MATCH_FLOOR: + return None, 0.0 + return best_id, best_ratio + + +def _claim_text_pool( + store: KBStore, *, exclude_proposal_id: str, exclude_claim_id: str | None, +) -> list[tuple[str, str]]: + pool = [ + (c.id, c.text) for c in store.list_claims() if c.id != exclude_claim_id + ] + pool += [ + (p.id, str(p.payload.get("text", ""))) + for p in store.list_proposals(ProposalStatus.PENDING) + if p.kind == ProposalKind.CLAIM and p.id != exclude_proposal_id + ] + return pool + + +def _embedding_hits_for_claim( + store: KBStore, proposal: Proposal, *, use_embeddings: bool, +) -> list[dict[str, Any]] | None: + """`find_similar_on_propose` hits, or None when the embedding path can't run. + + None means "no embedder available (or backend forced to heuristic)" — + callers fall back to a difflib heuristic. `[]` means the embedder ran + and genuinely found nothing similar. Every hit returned is, by that + function's own contract, at or above the near-duplicate threshold — + it's a duplicate detector, not a general similarity search. Used by + `duplication_risk` and `contradiction_risk`; deliberately NOT reused + for `fit` (see `_topical_fit_scores`). + """ + if not use_embeddings or proposal.kind != ProposalKind.CLAIM: + return None + text = str(proposal.payload.get("text", "")).strip() + if not text or _safe_embedder() is None: + return None + try: + from .embeddings.similarity import find_similar_on_propose + except ImportError: + return None + return find_similar_on_propose( + store, text, exclude_claim_id=proposal.payload.get("id"), + ) + + +def _topical_fit_scores(store: KBStore, proposal: Proposal, embedder: Any) -> list[float]: + """Cosine scores against the approved corpus, below the duplicate band. + + A near-duplicate hit is already penalized by `duplication_risk`; letting + it also inflate `fit` would let the two signals cancel each other out + for the exact-duplicate case. So this looks at a lower, wider band + (`min_score=0.3`) and excludes anything at or above the near-duplicate + threshold (`review.similarity_threshold`, default 0.95). + """ + text = str(proposal.payload.get("text", "")).strip() + if not text: + return [] + try: + from . import index_db + from .embeddings.similarity import similarity_threshold + + vec = embedder.encode(text) + dup_threshold = similarity_threshold(store) + hits = index_db.search_embedding( + store.kb_dir, query_vec=vec, kinds=("claim", "page"), limit=5, min_score=0.3, + ) + except Exception: + return [] + exclude_id = proposal.payload.get("id") + return [ + float(cos) for _kind, cid, _snip, cos in hits + if cid != exclude_id and cos < dup_threshold + ] + + +# --- signals ----------------------------------------------------------------- + + +def _signal_citation_quality(store: KBStore, proposal: Proposal) -> dict[str, Any]: + block = _payload_block_reason(store, proposal) + if block: + return {"score": 0.0, "reason": block} + if proposal.kind in (ProposalKind.CLAIM, ProposalKind.RELATION): + n = len(proposal.payload.get("evidence") or []) + if n == 0: + # Relations may legitimately have no evidence; claims can't reach + # here with n == 0 (Claim._at_least_one_citation already blocked). + return { + "score": 0.6, + "reason": "relation has no evidence citation (allowed, but weaker)", + } + score = min(1.0, 0.7 + 0.15 * (n - 1)) + return {"score": round(score, 4), "reason": f"{n} evidence citation(s) resolve cleanly"} + if proposal.kind == ProposalKind.PAGE: + sources = proposal.payload.get("sources") or [] + claims = proposal.payload.get("claims") or [] + if not sources and not claims: + return {"score": 0.5, "reason": "page has no source/claim citations"} + return { + "score": 1.0, + "reason": f"{len(sources)} source(s), {len(claims)} claim(s) resolve cleanly", + } + return {"score": 1.0, "reason": "entity payload resolves cleanly"} + + +def _signal_fit( + store: KBStore, proposal: Proposal, embedder: Any | None, +) -> dict[str, Any]: + entity_ids = _referenced_entity_ids(proposal) + known = {e.id for e in store.list_entities()} + overlap: float | None = None + if entity_ids: + overlap = sum(1 for e in entity_ids if e in known) / len(entity_ids) + + topical: float | None = None + if embedder is not None and proposal.kind == ProposalKind.CLAIM: + scores = _topical_fit_scores(store, proposal, embedder) + if scores: + topical = sum(scores) / len(scores) + + parts = [v for v in (overlap, topical) if v is not None] + if not parts: + return { + "score": 0.5, + "reason": "no referenced entities or approved-corpus signal; neutral fit", + } + bits = [] + if overlap is not None: + bits.append(f"{overlap:.0%} of referenced entities already known") + if topical is not None: + bits.append(f"mean topical similarity to approved corpus {topical:.2f}") + return {"score": round(sum(parts) / len(parts), 4), "reason": "; ".join(bits)} + + +def _duplication_risk_structural(store: KBStore, proposal: Proposal) -> dict[str, Any]: + if proposal.kind == ProposalKind.RELATION: + triple = ( + proposal.payload.get("source"), + proposal.payload.get("relation"), + proposal.payload.get("target"), + ) + for r in store.list_relations(): + if (r.source, r.relation.value, r.target) == triple: + return {"score": 1.0, "reason": f"identical relation already approved: {r.id}"} + for p in store.list_proposals(ProposalStatus.PENDING): + if p.kind != ProposalKind.RELATION or p.id == proposal.id: + continue + other = (p.payload.get("source"), p.payload.get("relation"), p.payload.get("target")) + if other == triple: + return {"score": 1.0, "reason": f"identical relation already pending: {p.id}"} + return {"score": 0.0, "reason": "no identical relation found"} + + if proposal.kind == ProposalKind.ENTITY: + name = str(proposal.payload.get("name", "")).strip() + pool = [(e.id, e.name) for e in store.list_entities()] + pool += [ + (p.id, str(p.payload.get("name", ""))) + for p in store.list_proposals(ProposalStatus.PENDING) + if p.kind == ProposalKind.ENTITY and p.id != proposal.id + ] + else: # PAGE + name = str(proposal.payload.get("title", "")).strip() + pool = [(pg.id, pg.title) for pg in store.list_pages()] + pool += [ + (p.id, str(p.payload.get("title", ""))) + for p in store.list_proposals(ProposalStatus.PENDING) + if p.kind == ProposalKind.PAGE and p.id != proposal.id + ] + + if not name: + return {"score": 0.0, "reason": "no name/title to compare"} + best_id, best_ratio = _best_fuzzy_match(name, pool) + if best_id is None: + return {"score": 0.0, "reason": "no similarly-named artifact found (heuristic backend)"} + return { + "score": round(best_ratio, 4), + "reason": f"name similarity {best_ratio:.2f} vs {best_id} (heuristic backend)", + } + + +def _signal_duplication_risk( + store: KBStore, proposal: Proposal, hits: list[dict[str, Any]] | None, +) -> dict[str, Any]: + if proposal.kind != ProposalKind.CLAIM: + return _duplication_risk_structural(store, proposal) + + text = str(proposal.payload.get("text", "")).strip() + if not text: + return {"score": 0.0, "reason": "no claim text to compare"} + + if hits is None: + pool = _claim_text_pool( + store, exclude_proposal_id=proposal.id, + exclude_claim_id=proposal.payload.get("id"), + ) + best_id, best_ratio = _best_fuzzy_match(text, pool) + if best_id is None: + return {"score": 0.0, "reason": "no near-duplicate claims found (heuristic backend)"} + return { + "score": round(best_ratio, 4), + "reason": f"text similarity {best_ratio:.2f} vs {best_id} (heuristic backend)", + } + + if not hits: + return {"score": 0.0, "reason": "no near-duplicate claims found (embedding backend)"} + top = max(hits, key=lambda w: w["cosine"]) + return { + "score": round(float(top["cosine"]), 4), + "reason": ( + f"cosine {top['cosine']:.2f} vs {top['artifact_kind']} " + f"{top['artifact_id']} (embedding backend)" + ), + } + + +def _signal_contradiction_risk( + store: KBStore, proposal: Proposal, hits: list[dict[str, Any]] | None, +) -> dict[str, Any]: + if proposal.kind != ProposalKind.CLAIM: + return {"score": 0.0, "reason": "contradiction risk is only assessed for claim proposals"} + + text = str(proposal.payload.get("text", "")).strip() + if not text: + return {"score": 0.0, "reason": "no claim text to compare"} + + entity_ids = set(proposal.payload.get("entities") or []) + neg = _has_negation(text) + + if hits is not None: + backend = "embedding" + candidates = [ + (h["artifact_id"], float(h["cosine"])) + for h in hits + if h.get("artifact_kind") == "claim" + ] + else: + backend = "heuristic" + pool = _claim_text_pool( + store, exclude_proposal_id=proposal.id, + exclude_claim_id=proposal.payload.get("id"), + ) + candidates = [ + (cid, ratio) + for cid, ratio in ( + (cid, difflib.SequenceMatcher(None, text.casefold(), ctext.casefold()).ratio()) + for cid, ctext in pool + ) + if ratio >= _CONTRADICTION_CANDIDATE_FLOOR + ] + + if not candidates: + return {"score": 0.0, "reason": f"no topically related claims found ({backend} backend)"} + + conflicts: list[tuple[str, float]] = [] + for cid, sim in candidates: + try: + claim = store.get_claim(cid) + except ArtifactNotFoundError: + continue # candidate is a pending proposal, not yet an approved claim + if entity_ids & set(claim.entities) and _has_negation(claim.text) != neg: + conflicts.append((cid, sim)) + + if not conflicts: + return { + "score": 0.0, + "reason": ( + f"{len(candidates)} related claim(s), " + f"no polarity conflict ({backend} backend)" + ), + } + top_id, top_sim = max(conflicts, key=lambda c: c[1]) + score = round(min(1.0, 0.5 + top_sim / 2), 4) + return { + "score": score, + "reason": ( + f"possible polarity conflict with {top_id} " + f"(similarity {top_sim:.2f}, {backend} backend)" + ), + } + + +# --- composite --------------------------------------------------------------- + + +def _composite_score(signals: dict[str, dict[str, Any]], weights: dict[str, float]) -> float: + goodness = { + "fit": signals["fit"]["score"], + "citation_quality": signals["citation_quality"]["score"], + "duplication_risk": 1.0 - signals["duplication_risk"]["score"], + "contradiction_risk": 1.0 - signals["contradiction_risk"]["score"], + } + total_weight = sum(weights.get(k, 0.0) for k in goodness) or 1.0 + raw = sum(goodness[k] * weights.get(k, 0.0) for k in goodness) / total_weight + return round(min(1.0, max(0.0, raw)), 4) + + +def _recommendation(score: float, signals: dict[str, dict[str, Any]]) -> str: + if signals["citation_quality"]["score"] == 0.0: + # A blocked payload can't be approved as-is (approve() would raise) — + # no composite score should be able to override that. + return "reject" + if score >= _APPROVE_THRESHOLD: + return "approve" + if score <= _REJECT_THRESHOLD: + return "reject" + return "needs-human" + + +def _rationale(recommendation: str, score: float, signals: dict[str, dict[str, Any]]) -> str: + parts = "; ".join(f"{name}: {sig['reason']}" for name, sig in signals.items()) + return f"{recommendation} (score {score:.2f}) — {parts}" + + +def score_proposal( + store: KBStore, proposal: Proposal, *, + weights: dict[str, float] | None = None, + use_embeddings: bool = True, +) -> dict[str, Any]: + """Compute the `_meta.vouch_triage` block for one pending proposal.""" + weights = weights or DEFAULT_WEIGHTS + hits = _embedding_hits_for_claim(store, proposal, use_embeddings=use_embeddings) + embedder = _safe_embedder() if use_embeddings else None + signals = { + "fit": _signal_fit(store, proposal, embedder), + "citation_quality": _signal_citation_quality(store, proposal), + "duplication_risk": _signal_duplication_risk(store, proposal, hits), + "contradiction_risk": _signal_contradiction_risk(store, proposal, hits), + } + score = _composite_score(signals, weights) + recommendation = _recommendation(score, signals) + return { + "recommendation": recommendation, + "score": score, + "signals": signals, + "rationale": _rationale(recommendation, score, signals), + } + + +def triage_pending( + store: KBStore, proposal_ids: list[str] | None = None, +) -> list[dict[str, Any]]: + """Score pending proposals (default: all of them) — read-only, advisory. + + Raises TriageError if `triage.enabled` isn't `true` in config.yaml. + """ + cfg = triage_cfg(store) + if not cfg.enabled: + raise TriageError( + "triage is disabled; set triage.enabled: true in .vouch/config.yaml to opt in" + ) + use_embeddings = cfg.backend != "heuristic" + + if proposal_ids: + proposals = [store.get_proposal(pid) for pid in proposal_ids] + proposals = [p for p in proposals if p.status == ProposalStatus.PENDING] + else: + proposals = store.list_proposals(ProposalStatus.PENDING) + + out: list[dict[str, Any]] = [] + for p in proposals: + result = p.model_dump(mode="json") + result.setdefault("_meta", {})["vouch_triage"] = score_proposal( + store, p, weights=cfg.weights, use_embeddings=use_embeddings, + ) + out.append(result) + return out diff --git a/src/vouch/volunteer_context.py b/src/vouch/volunteer_context.py index b1036e0f..33fd23c4 100644 --- a/src/vouch/volunteer_context.py +++ b/src/vouch/volunteer_context.py @@ -99,7 +99,7 @@ def session_query(sess: Session) -> str | None: def normalize_relevance(raw: float, backend: str, *, batch_max: float) -> float: - if backend in ("embedding", "hybrid"): + if backend == "embedding": return max(0.0, min(1.0, raw)) if batch_max <= 0.0: return 0.0 diff --git a/src/vouch/web/dual_solve_api.py b/src/vouch/web/dual_solve_api.py index 5b0e08df..c696cb94 100644 --- a/src/vouch/web/dual_solve_api.py +++ b/src/vouch/web/dual_solve_api.py @@ -68,9 +68,10 @@ def _serialize(job: DualSolveJob) -> dict[str, Any]: "candidates": [ {"engine": c.engine, "branch": c.branch, "ok": c.ok, "error": c.error, "changed_files": ds.changed_files(c.diff), - "diff": c.diff} + "log": c.log, "diff": c.diff} for c in job.candidates ], + "recommendation": ds.recommendation(job.candidates), "proposed_ids": list(job.proposed_ids), "kept_branch": job.kept_branch, "changed_files": ds.changed_files(kept.diff) if kept is not None else [], @@ -200,4 +201,5 @@ async def dual_solve_choose(req: _ChooseReq) -> dict[str, Any]: "kept_branch": job.kept_branch, "proposed_ids": ids, "changed_files": ds.changed_files(chosen.diff) if chosen is not None else [], + "recommendation": ds.recommendation(job.candidates), } diff --git a/src/vouch/web/static/dual_solve.css b/src/vouch/web/static/dual_solve.css index e94148a8..e77e4bf8 100644 --- a/src/vouch/web/static/dual_solve.css +++ b/src/vouch/web/static/dual_solve.css @@ -3,9 +3,12 @@ .ds-run input { flex: 1; } .ds-progress { background:#111; color:#ddd; padding:.5rem; white-space:pre-wrap; } .ds-error { color:#b00; } +.ds-recommendation { border:1px solid #ddd; padding:.5rem; background:#f8f8f8; } .ds-panes { display:grid; grid-template-columns:1fr 1fr; gap:1rem; } .ds-pane { border:1px solid #ccc; padding:.5rem; overflow:auto; } .ds-changed-files { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size:12px; margin:.25rem 0 .5rem; } +.ds-log { margin:.25rem 0 .75rem; } +.ds-log pre { max-height:220px; overflow:auto; white-space:pre-wrap; background:#111; color:#ddd; padding:.5rem; } .ds-file-head { font-weight:600; margin-top:.5rem; } .ds-pane pre { margin:0; font-size:12px; overflow-x:auto; } .ln-add { background:#e6ffed; display:block; } diff --git a/src/vouch/web/static/dual_solve.js b/src/vouch/web/static/dual_solve.js index fc0e9b85..a7772013 100644 --- a/src/vouch/web/static/dual_solve.js +++ b/src/vouch/web/static/dual_solve.js @@ -36,7 +36,7 @@ export default { const job = reactive({ id: null, status: "idle", progress: [], candidates: [], issue: null, error: null, kept_branch: null, proposed_ids: [], - changed_files: [], + changed_files: [], recommendation: null, }); function applyState(s) { @@ -64,6 +64,7 @@ export default { async function run() { job.progress = []; job.error = null; job.candidates = []; job.kept_branch = null; job.proposed_ids = []; job.changed_files = []; + job.recommendation = null; const r = await fetch("/dual-solve/run", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ @@ -105,6 +106,12 @@ export default {

#{{job.issue.number}} {{job.issue.title}}

{{ job.progress.join('\\n') }}

{{ job.error }}

+

+ recommendation: + {{job.recommendation.engine}} + no automatic pick + -- {{job.recommendation.reason}} +

@@ -113,6 +120,10 @@ export default {
  • {{f}}
+
+ {{c.engine}} log + 
{{c.log}}
+
{{f.path}}
{{l.text}}\\n
diff --git a/tests/fixtures/codex/rollout-basic.jsonl b/tests/fixtures/codex/rollout-basic.jsonl new file mode 100644 index 00000000..f4a51d9b --- /dev/null +++ b/tests/fixtures/codex/rollout-basic.jsonl @@ -0,0 +1,17 @@ +{"timestamp": "2026-07-01T09:00:00.000Z", "type": "session_meta", "payload": {"session_id": "0197aaaa-1111-7000-8000-000000000001", "id": "0197aaaa-1111-7000-8000-000000000001", "timestamp": "2026-07-01T09:00:00.000Z", "cwd": "/home/alice-example/projects/acme-app", "originator": "codex_exec", "cli_version": "0.142.0", "source": "exec"}} +{"timestamp": "2026-07-01T09:00:00.100Z", "type": "event_msg", "payload": {"type": "task_started", "turn_id": "0197aaaa-1111-7000-8000-0000000000t1"}} +{"timestamp": "2026-07-01T09:00:00.200Z", "type": "turn_context", "payload": {"turn_id": "0197aaaa-1111-7000-8000-0000000000t1", "cwd": "/home/alice-example/projects/acme-app", "model": "example-model"}} +{"timestamp": "2026-07-01T09:00:00.300Z", "type": "event_msg", "payload": {"type": "user_message", "message": "add a health endpoint to the acme api"}} +{"timestamp": "2026-07-01T09:00:01.000Z", "type": "response_item", "payload": {"type": "reasoning", "id": "rs_example", "summary": [], "encrypted_content": "opaque"}} +{"timestamp": "2026-07-01T09:00:02.000Z", "type": "response_item", "payload": {"type": "function_call", "id": "fc_1", "name": "exec_command", "arguments": "{\"cmd\":\"pytest -q\",\"workdir\":\"/home/alice-example/projects/acme-app\"}", "call_id": "call_1"}} +{"timestamp": "2026-07-01T09:00:03.000Z", "type": "response_item", "payload": {"type": "function_call_output", "call_id": "call_1", "output": "Process exited with code 1\nOutput:\n2 failed, 10 passed"}} +{"timestamp": "2026-07-01T09:00:04.000Z", "type": "response_item", "payload": {"type": "function_call", "id": "fc_2", "name": "exec_command", "arguments": "{\"cmd\":\"apply_patch <<'EOF'\\n*** Begin Patch\\n*** Update File: src/acme/api.py\\n@@\\n-old\\n+new\\n*** End Patch\\nEOF\",\"workdir\":\"/home/alice-example/projects/acme-app\"}", "call_id": "call_2"}} +{"timestamp": "2026-07-01T09:00:05.000Z", "type": "response_item", "payload": {"type": "function_call_output", "call_id": "call_2", "output": "Process exited with code 0\nOutput:\nDone!"}} +{"timestamp": "2026-07-01T09:00:06.000Z", "type": "response_item", "payload": {"type": "function_call", "id": "fc_3", "name": "update_plan", "arguments": "{\"plan\":[{\"step\":\"example\",\"status\":\"completed\"}]}", "call_id": "call_3"}} +{"timestamp": "2026-07-01T09:00:07.000Z", "type": "response_item", "payload": {"type": "function_call", "id": "fc_4", "name": "kb_search", "arguments": "{\"query\":\"health endpoint\"}", "call_id": "call_4"}} +{"timestamp": "2026-07-01T09:00:08.000Z", "type": "response_item", "payload": {"type": "function_call_output", "call_id": "call_4", "output": "{\"results\":[]}"}} +{"timestamp": "2026-07-01T09:00:09.000Z", "type": "response_item", "payload": {"type": "function_call", "id": "fc_5", "name": "exec_command", "arguments": "{\"cmd\":\"pytest -q\",\"workdir\":\"/home/alice-example/projects/acme-app\"}", "call_id": "call_5"}} +{"timestamp": "2026-07-01T09:00:10.000Z", "type": "response_item", "payload": {"type": "function_call_output", "call_id": "call_5", "output": "Process exited with code 0\nOutput:\n12 passed"}} +{"timestamp": "2026-07-01T09:00:11.000Z", "type": "world_state", "payload": {"unknown_future_field": true}} +{"timestamp": "2026-07-01T09:00:12.000Z", "type": "event_msg", "payload": {"type": "agent_message", "message": "added the endpoint and the tests pass"}} +{"timestamp": "2026-07-01T09:00:13.000Z", "type": "event_msg", "payload": {"type": "task_complete", "turn_id": "0197aaaa-1111-7000-8000-0000000000t1", "last_agent_message": "added the endpoint and the tests pass"}} diff --git a/tests/fixtures/codex/rollout-no-meta.jsonl b/tests/fixtures/codex/rollout-no-meta.jsonl new file mode 100644 index 00000000..cc1d9252 --- /dev/null +++ b/tests/fixtures/codex/rollout-no-meta.jsonl @@ -0,0 +1,2 @@ +{"timestamp": "2026-07-01T09:00:00.300Z", "type": "event_msg", "payload": {"type": "user_message", "message": "hello"}} +{"timestamp": "2026-07-01T09:00:02.000Z", "type": "response_item", "payload": {"type": "function_call", "id": "fc_1", "name": "exec_command", "arguments": "{\"cmd\":\"ls\"}", "call_id": "call_1"}} diff --git a/tests/test_capabilities.py b/tests/test_capabilities.py index 736b20ba..49c29c99 100644 --- a/tests/test_capabilities.py +++ b/tests/test_capabilities.py @@ -2,6 +2,11 @@ from __future__ import annotations +import json +from pathlib import Path + +import pytest + from vouch import capabilities from vouch.jsonl_server import HANDLERS @@ -15,3 +20,87 @@ def test_capabilities_matches_jsonl_handlers() -> None: f"missing handlers={declared - implemented}, " f"missing capabilities={implemented - declared}" ) + + +# --- host_compat drift detection (#237) ----------------------------------- +# +# vouch declares openclaw.compat.pluginApi in openclaw.plugin.json. The same +# value must surface in kb.capabilities so non-OpenClaw clients can detect +# compat without parsing the manifest. These tests fail CI with a clear +# message if the two declarations drift apart. + +_MANIFEST_PATH = ( + Path(__file__).resolve().parent.parent / "openclaw.plugin.json" +) + + +def _manifest_plugin_api() -> str: + manifest = json.loads(_MANIFEST_PATH.read_text(encoding="utf-8")) + return manifest["openclaw"]["compat"]["pluginApi"] + + +def test_capabilities_host_compat_matches_openclaw_manifest() -> None: + """kb.capabilities.host_compat.openclaw.pluginApi must equal the + pluginApi range declared in openclaw.plugin.json. A bump in one file + without the other is exactly the "host compat drift" #237 asks CI to + catch.""" + caps = capabilities.capabilities() + manifest_range = _manifest_plugin_api() + capabilities_range = caps.host_compat.get("openclaw", {}).get("pluginApi") + assert capabilities_range == manifest_range, ( + f"host compat drift: openclaw.plugin.json declares pluginApi=" + f"{manifest_range!r} but kb.capabilities.host_compat reports " + f"{capabilities_range!r}. Keep both in sync." + ) + + +def test_capabilities_host_compat_present_and_nonempty() -> None: + """host_compat must not silently degrade to {} when the manifest is + readable -- that would defeat the drift check above by making both + sides agree on "missing" rather than catching real drift.""" + caps = capabilities.capabilities() + assert "openclaw" in caps.host_compat + assert "pluginApi" in caps.host_compat["openclaw"] + assert caps.host_compat["openclaw"]["pluginApi"].strip() != "" + + +def test_load_host_compat_returns_empty_on_missing_manifest( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path, +) -> None: + """_load_host_compat must degrade gracefully (empty dict, no raise) if + the manifest is absent -- e.g. installed as a standalone wheel without + openclaw.plugin.json packaged alongside it.""" + monkeypatch.setattr( + capabilities, "_PLUGIN_MANIFEST_PATH", tmp_path / "does-not-exist.json" + ) + assert capabilities._load_host_compat() == {} + + +def test_load_host_compat_returns_empty_on_malformed_manifest( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path, +) -> None: + """A malformed manifest must not crash capabilities() -- it's reporting + diagnostic info, not validating the install.""" + bad = tmp_path / "openclaw.plugin.json" + bad.write_text("{not valid json", encoding="utf-8") + monkeypatch.setattr(capabilities, "_PLUGIN_MANIFEST_PATH", bad) + assert capabilities._load_host_compat() == {} + + +def test_mcp_tools_match_methods() -> None: + """Every MCP kb_* tool maps to a capabilities method and vice-versa. + + Closes the MCP half of the 3-surface parity invariant that the JSONL + check above did not cover. Uses the unfiltered server object (profiles + apply only in run_stdio). + """ + from vouch.server import mcp + + tool_names = {n for n in mcp._tool_manager._tools if n.startswith("kb_")} + as_methods = {"kb." + n.split("_", 1)[1] for n in tool_names} + declared = set(capabilities.METHODS) + assert as_methods == declared, ( + f"mcp/methods mismatch: " + f"missing tools={declared - as_methods}, " + f"undeclared tools={as_methods - declared}" + ) diff --git a/tests/test_codex_adapter_load_real.py b/tests/test_codex_adapter_load_real.py new file mode 100644 index 00000000..de230b49 --- /dev/null +++ b/tests/test_codex_adapter_load_real.py @@ -0,0 +1,199 @@ +"""Tier-2 e2e: the real Codex CLI reads the installed adapter (#389). + +The unit tests in test_install_adapter.py assert what the installer writes; +nothing there proves the real codex CLI accepts it — the old T1 silent-skip +behavior (installer no-op whenever `.codex/config.toml` existed) is exactly +the class of bug only a live gate catches. This suite closes that gap, +following the pattern tests/test_openclaw_plugin_load_real.py set: install +the adapter into a temp project at the highest shipped tier, mark the +project trusted inside an isolated ``CODEX_HOME``, and assert through codex +itself (``codex mcp list --json``) that the vouch server is visible — next +to a pre-seeded unrelated server on the merge path, and alone on the +fresh-install path. + +Assertions target the observable contract (server listed, config parses, +snippet present), not codex internals, so codex version bumps shouldn't +break the suite. Skips when the ``codex`` CLI is not on PATH (e.g. GitHub +CI). Every codex invocation runs with a throwaway ``CODEX_HOME`` and a temp +project cwd, so the user's real ``~/.codex`` is never touched; listing +configured servers needs no network and no credentials. +""" + +from __future__ import annotations + +import json +import os +import shutil +import subprocess +import tomllib +from pathlib import Path + +import pytest + +from vouch.install_adapter import install + +CODEX = shutil.which("codex") + +pytestmark = pytest.mark.skipif(CODEX is None, reason="codex CLI not on PATH") + + +def _run_codex( + env: dict[str, str], cwd: Path, *args: str +) -> subprocess.CompletedProcess[str]: + assert CODEX is not None + return subprocess.run( + [CODEX, *args], + env=env, + cwd=cwd, + capture_output=True, + text=True, + timeout=120, + check=False, + ) + + +def _isolated_env(home: Path) -> dict[str, str]: + home.mkdir(parents=True, exist_ok=True) + return {**os.environ, "CODEX_HOME": str(home)} + + +def _trust(home: Path, project: Path) -> None: + # Codex loads project-scoped .codex/ layers only for trusted projects; + # this is the non-interactive equivalent of answering the trust prompt. + # The path is a TOML quoted key, so escape it with json.dumps (TOML + # basic strings share JSON's escaping) — a backslash or quote in the + # path would otherwise produce invalid TOML. + key = json.dumps(str(project)) + with (home / "config.toml").open("a", encoding="utf-8") as fh: + fh.write(f'[projects.{key}]\ntrust_level = "trusted"\n') + + +def _mcp_servers(env: dict[str, str], cwd: Path) -> list[dict]: + result = _run_codex(env, cwd, "mcp", "list", "--json") + assert result.returncode == 0, ( + f"codex mcp list failed.\nstdout: {result.stdout}\nstderr: {result.stderr}" + ) + # `codex mcp list --json` may emit a one-time notice before the JSON, + # so try a straight parse first and fall back to slicing from the first + # bracket; on failure surface stdout/stderr so the reason is actionable. + try: + servers = json.loads(result.stdout) + except json.JSONDecodeError: + start = result.stdout.find("[") + if start == -1: + raise AssertionError( + f"codex mcp list produced no JSON array.\n" + f"stdout: {result.stdout!r}\nstderr: {result.stderr!r}" + ) from None + try: + servers = json.loads(result.stdout[start:]) + except json.JSONDecodeError as e: + raise AssertionError( + f"codex mcp list JSON did not parse: {e}\n" + f"stdout: {result.stdout!r}\nstderr: {result.stderr!r}" + ) from e + assert isinstance(servers, list) + return servers + + +def test_merge_install_is_visible_to_codex(tmp_path: Path) -> None: + """The #384 regression at the live level: a project where codex is + already configured must end up with vouch wired next to the user's + existing server, as seen by codex itself.""" + home = tmp_path / "codex-home" + project = tmp_path / "project" + (project / ".codex").mkdir(parents=True) + (project / ".codex" / "config.toml").write_text( + 'model = "gpt-5"\n\n[mcp_servers.other]\ncommand = "other-server"\n', + encoding="utf-8", + ) + + install("codex", target=project, tier="T4") + + # the merged file is still valid toml with both servers side by side + merged = tomllib.loads( + (project / ".codex" / "config.toml").read_text(encoding="utf-8") + ) + assert merged["model"] == "gpt-5" + assert set(merged["mcp_servers"]) == {"other", "vouch"} + + env = _isolated_env(home) + _trust(home, project) + servers = {s["name"]: s for s in _mcp_servers(env, project)} + assert "vouch" in servers, f"vouch not listed: {sorted(servers)}" + assert "other" in servers, "user's pre-existing server disappeared" + transport = servers["vouch"]["transport"] + assert transport["command"] == "vouch" + assert transport["args"] == ["serve"] + + +def test_fresh_install_is_visible_to_codex(tmp_path: Path) -> None: + home = tmp_path / "codex-home" + project = tmp_path / "project" + project.mkdir() + + install("codex", target=project, tier="T4") + + env = _isolated_env(home) + _trust(home, project) + names = [s["name"] for s in _mcp_servers(env, project)] + assert names == ["vouch"], names + + +def test_untrusted_project_config_stays_inert(tmp_path: Path) -> None: + """Codex ignores project-scoped config for untrusted projects — the + install must not leak into a session the user never trusted.""" + home = tmp_path / "codex-home" + project = tmp_path / "project" + project.mkdir() + install("codex", target=project, tier="T4") + + env = _isolated_env(home) # note: no _trust() call + names = [s["name"] for s in _mcp_servers(env, project)] + assert "vouch" not in names, names + + +def test_full_tier_artifacts_present(tmp_path: Path) -> None: + """The T4 install ships every codex-readable surface: merged config, + fenced AGENTS.md, the nine skills, and the Stop hook.""" + home = tmp_path / "codex-home" + project = tmp_path / "project" + project.mkdir() + install("codex", target=project, tier="T4") + + agents = (project / "AGENTS.md").read_text(encoding="utf-8") + assert "" in agents + assert "VOUCH_AGENT=codex" in agents.replace("`", "") + + assert (project / ".codex" / "skills" / "vouch-recall" / "SKILL.md").is_file() + + hooks = json.loads( + (project / ".codex" / "hooks.json").read_text(encoding="utf-8") + ) + cmds = [h["command"] for g in hooks["hooks"]["Stop"] for h in g["hooks"]] + assert "vouch capture ingest-codex --hook" in cmds + + # and codex still accepts the whole tree + env = _isolated_env(home) + _trust(home, project) + assert [s["name"] for s in _mcp_servers(env, project)] == ["vouch"] + + +def test_nothing_written_outside_project_and_home(tmp_path: Path) -> None: + """#179 invariant at the live level: after an install plus a codex + invocation, the throwaway home holds only what we put there and the + project holds only adapter artifacts — the real ~/.codex is never in + play because CODEX_HOME is pinned for every invocation.""" + home = tmp_path / "codex-home" + project = tmp_path / "project" + project.mkdir() + result = install("codex", target=project, tier="T4") + for rel in (*result.written, *result.appended, *result.merged): + assert (project / rel).resolve().is_relative_to(project.resolve()) + + env = _isolated_env(home) + _trust(home, project) + _mcp_servers(env, project) + # the trust entry is the only file vouch's test wrote into the home; + # codex may add its own state (caches, logs) but only under that home. + assert (home / "config.toml").is_file() diff --git a/tests/test_codex_rollout.py b/tests/test_codex_rollout.py new file mode 100644 index 00000000..2969008c --- /dev/null +++ b/tests/test_codex_rollout.py @@ -0,0 +1,461 @@ +"""`vouch capture ingest-codex` — codex rollout parsing + review-gated +ingest (vouchdev/vouch#387). + +Codex persists sessions as rollout jsonl files instead of emitting live +hooks. The parser maps rollout records into capture observations; the +ingest path reuses the existing rollup so a codex session yields the same +kind of PENDING page proposal a claude session does, deduped on the +rollout's session id. Fixtures use placeholder data only (alice-example). +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest +from click.testing import CliRunner + +from vouch import codex_rollout as cr +from vouch.cli import cli +from vouch.models import ProposalKind, ProposalStatus +from vouch.storage import KBStore + +FIXTURES = Path(__file__).parent / "fixtures" / "codex" +BASIC = FIXTURES / "rollout-basic.jsonl" +NO_META = FIXTURES / "rollout-no-meta.jsonl" +BASIC_SESSION = "0197aaaa-1111-7000-8000-000000000001" + + +@pytest.fixture +def store(tmp_path: Path) -> KBStore: + return KBStore.init(tmp_path) + + +# --- parser ---------------------------------------------------------------- + + +def test_parse_basic_rollout_extracts_session_meta() -> None: + session = cr.parse_rollout(BASIC) + assert session.session_id == BASIC_SESSION + assert session.cwd == "/home/alice-example/projects/acme-app" + assert session.started_at == "2026-07-01T09:00:00.000Z" + assert session.first_prompt == "add a health endpoint to the acme api" + + +def test_parse_maps_tool_calls_to_observations() -> None: + session = cr.parse_rollout(BASIC) + summaries = [o["summary"] for o in session.observations] + # exec_command with a non-zero exit is marked failed, like live capture + assert "Command failed: pytest -q" in summaries + # ...and the same command succeeding later reads as a plain run + assert "Ran: pytest -q" in summaries + # an apply_patch heredoc through the shell surfaces as a file edit + edit = next(o for o in session.observations if o["tool"] == "Edit") + assert edit["files"] == ["src/acme/api.py"] + assert edit["summary"] == "Edited api.py" + # mcp/custom tools keep their own name + assert any(o["tool"] == "kb_search" for o in session.observations) + # session mechanics (update_plan) are not observations + assert not any("update_plan" in o["tool"] for o in session.observations) + + +def test_parse_keeps_bash_cmd_for_notable_commands() -> None: + session = cr.parse_rollout(BASIC) + bash = [o for o in session.observations if o["tool"] == "Bash"] + assert all(o.get("cmd") for o in bash) + + +def test_patch_observation_verbs() -> None: + add = cr._patch_observation("*** Add File: src/new.py\n") + assert add is not None and add["summary"] == "Created new.py" + delete = cr._patch_observation("*** Delete File: src/old.py\n") + assert delete is not None and delete["summary"] == "Deleted old.py" + update = cr._patch_observation("*** Update File: src/mod.py\n") + assert update is not None and update["summary"] == "Edited mod.py" + mixed = cr._patch_observation("*** Add File: a.py\n*** Delete File: b.py\n") + assert mixed is not None and mixed["summary"] == "Edited 2 files" + + +def test_parse_tolerates_unknown_record_types() -> None: + # rollout-basic.jsonl includes a world_state record and a reasoning + # item; both must be skipped, not fatal. + session = cr.parse_rollout(BASIC) + assert len(session.observations) == 4 + + +def test_parse_without_session_meta_is_actionable_error() -> None: + with pytest.raises(cr.CodexRolloutError, match="session_meta"): + cr.parse_rollout(NO_META) + + +def test_parse_missing_file_is_actionable_error(tmp_path: Path) -> None: + with pytest.raises(cr.CodexRolloutError, match="cannot read"): + cr.parse_rollout(tmp_path / "nope.jsonl") + + +def test_parse_zstd_compressed_is_actionable_error(tmp_path: Path) -> None: + frame = tmp_path / "rollout-2026-07-01T09-00-00-x.jsonl" + frame.write_bytes(b"\x28\xb5\x2f\xfd" + b"\x00" * 16) + with pytest.raises(cr.CodexRolloutError, match="zstd"): + cr.parse_rollout(frame) + + +# --- ingest ---------------------------------------------------------------- + + +def test_ingest_files_one_pending_page(store: KBStore) -> None: + result = cr.ingest_rollout(store, BASIC, generated_at="2026-07-01T10:00:00Z") + pid = result["summary_proposal_id"] + assert pid is not None + assert result["captured"] == 4 + pend = store.list_proposals(ProposalStatus.PENDING) + assert [p.id for p in pend] == [pid] + pr = pend[0] + assert pr.kind == ProposalKind.PAGE + assert pr.proposed_by == cr.CODEX_ACTOR + assert pr.session_id == BASIC_SESSION + assert pr.payload["type"] == "session" + body = pr.payload["body"] + assert "src/acme/api.py" in body + assert "pytest -q" in body + assert pr.payload["title"].startswith("session: add a health endpoint") + assert "[acme-app]" in pr.payload["title"] + + +def test_ingest_respects_vouch_agent_env(store: KBStore, monkeypatch) -> None: + monkeypatch.setenv("VOUCH_AGENT", "codex-nightly") + cr.ingest_rollout(store, BASIC) + pend = store.list_proposals(ProposalStatus.PENDING) + assert pend[0].proposed_by == "codex-nightly" + + +def test_reingest_same_session_is_noop(store: KBStore) -> None: + first = cr.ingest_rollout(store, BASIC) + second = cr.ingest_rollout(store, BASIC) + assert second["skipped"] == "already-ingested" + assert second["summary_proposal_id"] == first["summary_proposal_id"] + assert len(store.list_proposals(None)) == 1 + + +def test_ingest_below_min_files_nothing(store: KBStore) -> None: + store.config_path.write_text( + "capture:\n min_observations: 99\n", encoding="utf-8" + ) + result = cr.ingest_rollout(store, BASIC) + assert result["skipped"] == "below-min" + assert result["summary_proposal_id"] is None + assert store.list_proposals(None) == [] + + +def test_ingest_noop_when_capture_disabled(store: KBStore) -> None: + store.config_path.write_text( + "capture:\n enabled: false\n", encoding="utf-8" + ) + result = cr.ingest_rollout(store, BASIC) + assert result["skipped"] == "disabled" + assert store.list_proposals(None) == [] + + +def test_ingest_never_writes_approved_content(store: KBStore) -> None: + """The review gate stays intact: ingest files a proposal, not a page.""" + cr.ingest_rollout(store, BASIC) + assert store.list_pages() == [] + + +# --- per-turn re-ingest: refresh the pending proposal (vouchdev/vouch#388) -- + + +def _grown_copy(tmp_path: Path) -> Path: + """BASIC plus one more tool call, same session id — a later turn.""" + grown = tmp_path / f"rollout-2026-07-01T09-30-00-{BASIC_SESSION}.jsonl" + extra = { + "timestamp": "2026-07-01T09:30:00.000Z", + "type": "response_item", + "payload": { + "type": "function_call", "name": "exec_command", + "arguments": json.dumps({"cmd": "ruff check src"}), + "call_id": "call_extra", + }, + } + grown.write_text( + BASIC.read_text(encoding="utf-8") + json.dumps(extra) + "\n", + encoding="utf-8", + ) + return grown + + +def test_reingest_grown_session_updates_pending_in_place( + store: KBStore, tmp_path: Path +) -> None: + first = cr.ingest_rollout(store, BASIC) + pid = first["summary_proposal_id"] + second = cr.ingest_rollout(store, _grown_copy(tmp_path)) + assert second["updated"] is True + assert second["summary_proposal_id"] == pid + proposals = store.list_proposals(None) + assert len(proposals) == 1 # refreshed, not duplicated + assert "ruff check src" in proposals[0].payload["body"] + assert proposals[0].status == ProposalStatus.PENDING + + +def test_reingest_decided_session_stays_decided( + store: KBStore, tmp_path: Path +) -> None: + """A proposal the human already reviewed is history — a later turn must + not resurrect or mutate it.""" + from vouch.proposals import approve + + first = cr.ingest_rollout(store, BASIC) + approve(store, first["summary_proposal_id"], approved_by="alice-example") + result = cr.ingest_rollout(store, _grown_copy(tmp_path)) + assert result["skipped"] == "already-ingested" + assert len(store.list_proposals(ProposalStatus.PENDING)) == 0 + + +def test_reingest_update_lands_in_audit_log(store: KBStore, tmp_path: Path) -> None: + from vouch import audit + + cr.ingest_rollout(store, BASIC) + cr.ingest_rollout(store, _grown_copy(tmp_path)) + events = [e.event for e in audit.read_events(store.kb_dir)] + assert "proposal.page.update" in events + + +# --- hook wire (--hook): codex Stop event ------------------------------------ + + +def test_ingest_hook_payload_files_proposal(store: KBStore) -> None: + result = cr.ingest_hook_payload( + store, + {"session_id": BASIC_SESSION, "transcript_path": str(BASIC), + "hook_event_name": "Stop", "cwd": str(store.kb_dir.parent)}, + ) + assert result is not None + assert result["summary_proposal_id"] + + +def test_ingest_hook_payload_resolves_by_session_id( + store: KBStore, tmp_path: Path +) -> None: + home = tmp_path / "codex-home" + day = home / "sessions" / "2026" / "07" / "01" + day.mkdir(parents=True) + rollout = day / f"rollout-2026-07-01T09-00-00-{BASIC_SESSION}.jsonl" + rollout.write_text(BASIC.read_text(encoding="utf-8"), encoding="utf-8") + result = cr.ingest_hook_payload( + store, {"session_id": BASIC_SESSION}, codex_home=home + ) + assert result is not None + assert result["summary_proposal_id"] + + +def test_ingest_hook_payload_never_raises(store: KBStore, tmp_path: Path) -> None: + assert cr.ingest_hook_payload(None, {"session_id": "x"}) is None + assert cr.ingest_hook_payload(store, {}) is None + assert ( + cr.ingest_hook_payload( + store, {"session_id": "no-such-session"}, + codex_home=tmp_path / "empty", + ) + is None + ) + + +def test_cli_hook_mode_files_proposal(store: KBStore, monkeypatch) -> None: + monkeypatch.chdir(store.kb_dir.parent) + payload = json.dumps({ + "session_id": BASIC_SESSION, + "transcript_path": str(BASIC), + "hook_event_name": "Stop", + }) + res = CliRunner().invoke( + cli, ["capture", "ingest-codex", "--hook"], input=payload + ) + assert res.exit_code == 0, res.output + assert len(store.list_proposals(ProposalStatus.PENDING)) == 1 + + +def test_cli_hook_mode_exits_zero_on_garbage(store: KBStore, monkeypatch) -> None: + monkeypatch.chdir(store.kb_dir.parent) + res = CliRunner().invoke( + cli, ["capture", "ingest-codex", "--hook"], input="{ not json" + ) + assert res.exit_code == 0, res.output + + +def test_cli_hook_mode_exits_zero_without_kb(tmp_path: Path, monkeypatch) -> None: + monkeypatch.chdir(tmp_path) # no .vouch anywhere above tmp_path + payload = json.dumps({"session_id": BASIC_SESSION, "transcript_path": str(BASIC)}) + res = CliRunner().invoke( + cli, ["capture", "ingest-codex", "--hook"], input=payload + ) + assert res.exit_code == 0, res.output + + +def test_find_rollout_by_session_id(tmp_path: Path) -> None: + home = tmp_path / "codex-home" + day = home / "sessions" / "2026" / "07" / "01" + day.mkdir(parents=True) + target = day / "rollout-2026-07-01T09-00-00-sess-42.jsonl" + target.write_text("{}", encoding="utf-8") + (day / "rollout-2026-07-01T10-00-00-sess-43.jsonl").write_text( + "{}", encoding="utf-8" + ) + assert cr.find_rollout_by_session_id("sess-42", codex_home=home) == target + assert cr.find_rollout_by_session_id("sess-99", codex_home=home) is None + + +def test_find_rollout_by_session_id_treats_glob_chars_literally( + tmp_path: Path, +) -> None: + """A session id from a hook payload is matched as a literal suffix, so + glob metacharacters can't widen the search to unrelated rollouts.""" + home = tmp_path / "codex-home" + day = home / "sessions" / "2026" / "07" / "01" + day.mkdir(parents=True) + (day / "rollout-2026-07-01T09-00-00-sess-42.jsonl").write_text( + "{}", encoding="utf-8" + ) + # `*` must not match the real session above. + assert cr.find_rollout_by_session_id("sess-*", codex_home=home) is None + + +# --- --latest resolution ---------------------------------------------------- + + +def _write_rollout(sessions: Path, day: str, stamp: str, sid: str, cwd: str) -> Path: + d = sessions / day + d.mkdir(parents=True, exist_ok=True) + path = d / f"rollout-{stamp}-{sid}.jsonl" + meta = { + "timestamp": f"{day.replace('/', '-')}T00:00:00.000Z", + "type": "session_meta", + "payload": {"id": sid, "session_id": sid, "cwd": cwd}, + } + path.write_text(json.dumps(meta) + "\n", encoding="utf-8") + return path + + +def test_find_latest_rollout_matches_project_cwd(tmp_path: Path) -> None: + home = tmp_path / "codex-home" + sessions = home / "sessions" + project = tmp_path / "proj" + project.mkdir() + other = "/home/alice-example/elsewhere" + _write_rollout(sessions, "2026/07/01", "2026-07-01T08-00-00", "aaa", str(project)) + newest_match = _write_rollout( + sessions, "2026/07/02", "2026-07-02T08-00-00", "bbb", str(project) + ) + _write_rollout(sessions, "2026/07/03", "2026-07-03T08-00-00", "ccc", other) + found = cr.find_latest_rollout(project, codex_home=home) + assert found == newest_match + + +def test_find_latest_rollout_none_when_no_match(tmp_path: Path) -> None: + home = tmp_path / "codex-home" + _write_rollout(home / "sessions", "2026/07/01", "2026-07-01T08-00-00", + "aaa", "/home/alice-example/elsewhere") + project = tmp_path / "proj" + project.mkdir() + assert cr.find_latest_rollout(project, codex_home=home) is None + + +def test_find_latest_rollout_none_without_sessions_dir(tmp_path: Path) -> None: + assert cr.find_latest_rollout(tmp_path, codex_home=tmp_path / "nope") is None + + +# --- CLI surface ------------------------------------------------------------ + + +def test_cli_ingest_codex_files_proposal(store: KBStore, monkeypatch) -> None: + monkeypatch.chdir(store.kb_dir.parent) + res = CliRunner().invoke(cli, ["capture", "ingest-codex", str(BASIC)]) + assert res.exit_code == 0, res.output + out = json.loads(res.output) + assert out["summary_proposal_id"] + assert out["session_id"] == BASIC_SESSION + + +def test_cli_ingest_codex_reingest_reports_noop(store: KBStore, monkeypatch) -> None: + monkeypatch.chdir(store.kb_dir.parent) + runner = CliRunner() + runner.invoke(cli, ["capture", "ingest-codex", str(BASIC)]) + res = runner.invoke(cli, ["capture", "ingest-codex", str(BASIC)]) + assert res.exit_code == 0, res.output + assert json.loads(res.output)["skipped"] == "already-ingested" + + +def test_cli_ingest_codex_malformed_is_clean_error( + store: KBStore, monkeypatch +) -> None: + monkeypatch.chdir(store.kb_dir.parent) + res = CliRunner().invoke(cli, ["capture", "ingest-codex", str(NO_META)]) + assert res.exit_code != 0 + assert "Traceback" not in res.output + assert "Error:" in res.output + assert store.list_proposals(None) == [] + + +def test_cli_ingest_codex_requires_exactly_one_source( + store: KBStore, monkeypatch +) -> None: + monkeypatch.chdir(store.kb_dir.parent) + runner = CliRunner() + neither = runner.invoke(cli, ["capture", "ingest-codex"]) + assert neither.exit_code != 0 + assert "exactly one" in neither.output + both = runner.invoke(cli, ["capture", "ingest-codex", str(BASIC), "--latest"]) + assert both.exit_code != 0 + assert "exactly one" in both.output + + +def test_cli_ingest_codex_latest_resolves_for_project( + store: KBStore, tmp_path: Path, monkeypatch +) -> None: + project = store.kb_dir.parent + home = tmp_path / "codex-home" + sid = "0197bbbb-2222-7000-8000-000000000002" + path = _write_rollout( + home / "sessions", "2026/07/02", "2026-07-02T09-00-00", sid, + str(project.resolve()), + ) + # give the rollout enough activity to clear min_observations + with path.open("a", encoding="utf-8") as fh: + for i in range(3): + fh.write(json.dumps({ + "timestamp": "2026-07-02T09:00:01.000Z", + "type": "response_item", + "payload": { + "type": "function_call", "name": "exec_command", + "arguments": json.dumps({"cmd": f"echo step-{i}"}), + "call_id": f"call_{i}", + }, + }) + "\n") + monkeypatch.chdir(project) + res = CliRunner().invoke( + cli, ["capture", "ingest-codex", "--latest", "--codex-home", str(home)], + ) + assert res.exit_code == 0, res.output + assert json.loads(res.output)["session_id"] == sid + + +def test_cli_ingest_codex_latest_no_rollout_is_clean_error( + store: KBStore, tmp_path: Path, monkeypatch +) -> None: + monkeypatch.chdir(store.kb_dir.parent) + res = CliRunner().invoke( + cli, + ["capture", "ingest-codex", "--latest", "--codex-home", + str(tmp_path / "empty-home")], + ) + assert res.exit_code != 0 + assert "no codex rollout found" in res.output + + +def test_fixtures_use_placeholder_data_only() -> None: + """Privacy rule: fixture rollouts must not carry real paths or names.""" + for fixture in FIXTURES.glob("*.jsonl"): + text = fixture.read_text(encoding="utf-8") + assert "alice-example" in text or "session_meta" not in text + assert "/home/a/" not in text diff --git a/tests/test_diff.py b/tests/test_diff.py index 4189d806..f26c063a 100644 --- a/tests/test_diff.py +++ b/tests/test_diff.py @@ -7,8 +7,11 @@ import pytest from click.testing import CliRunner +from vouch import audit +from vouch.capabilities import capabilities from vouch.cli import cli from vouch.diff import ArtifactDiff, DiffError, diff_artifacts +from vouch.jsonl_server import HANDLERS, handle_request from vouch.models import Claim, ClaimStatus, Page from vouch.storage import KBStore @@ -80,6 +83,36 @@ def test_diff_mismatched_kinds_raises(store: KBStore) -> None: diff_artifacts(store, "c1", "p1") +def test_diff_omitted_new_id_resolves_via_superseded_by(store: KBStore) -> None: + _claim(store, "c2", text="new wording") + _claim(store, "c1", text="old wording", superseded_by="c2") + d = diff_artifacts(store, "c1") + assert d.new_id == "c2" + assert any(line.startswith("+new wording") for line in d.text_diff) + + +def test_diff_omitted_new_id_without_successor_raises(store: KBStore) -> None: + _claim(store, "c1") + with pytest.raises(DiffError, match="has not been superseded"): + diff_artifacts(store, "c1") + + +def test_diff_omitted_new_id_for_page_raises(store: KBStore) -> None: + store.put_page(Page(id="p1", title="P", body="b")) + with pytest.raises(DiffError, match="pages have no successor pointer"): + diff_artifacts(store, "p1") + + +def test_diff_read_only_writes_no_audit_event_or_proposal(store: KBStore) -> None: + _claim(store, "c1", text="old") + _claim(store, "c2", text="new") + before = list(audit.read_events(store.kb_dir)) + diff_artifacts(store, "c1", "c2") + after = list(audit.read_events(store.kb_dir)) + assert after == before + assert store.list_proposals() == [] + + # --- CLI ------------------------------------------------------------------ @@ -127,3 +160,68 @@ def test_cli_diff_unknown_id_clean_error( assert res.exit_code != 0 assert "Traceback" not in res.output assert "unknown artifact: nope" in res.output + + +def test_cli_diff_omitted_new_id_resolves_successor( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.chdir(store.root) + _claim(store, "c2", text="new") + _claim(store, "c1", text="old", superseded_by="c2") + res = CliRunner().invoke(cli, ["diff", "c1"]) + assert res.exit_code == 0, res.output + assert "diff claim c1 → c2" in res.output + + +def test_cli_diff_omitted_new_id_clean_error( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.chdir(store.root) + _claim(store, "c1") + res = CliRunner().invoke(cli, ["diff", "c1"]) + assert res.exit_code != 0 + assert "Traceback" not in res.output + assert "has not been superseded" in res.output + + +# --- kb.* RPC surface ------------------------------------------------------- + + +def test_diff_method_in_capabilities() -> None: + methods = set(capabilities().methods) + assert "kb.diff" in methods + assert set(capabilities().methods) == set(HANDLERS.keys()) + + +def test_kb_diff_over_jsonl(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.chdir(store.root) + _claim(store, "c1", status=ClaimStatus.WORKING) + _claim(store, "c2", status=ClaimStatus.STABLE) + resp = handle_request( + {"id": "1", "method": "kb.diff", "params": {"old_id": "c1", "new_id": "c2"}} + ) + assert resp["ok"] is True, resp + assert resp["result"]["kind"] == "claim" + changed = {c["field"]: (c["old"], c["new"]) for c in resp["result"]["changes"]} + assert changed["status"] == ("working", "stable") + + +def test_kb_diff_missing_param_over_jsonl( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.chdir(store.root) + resp = handle_request({"id": "2", "method": "kb.diff", "params": {}}) + assert resp["ok"] is False + assert resp["error"]["code"] == "missing_param" + + +def test_kb_diff_unknown_id_over_jsonl( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.chdir(store.root) + _claim(store, "c1") + resp = handle_request( + {"id": "3", "method": "kb.diff", "params": {"old_id": "c1", "new_id": "nope"}} + ) + assert resp["ok"] is False + assert resp["error"]["code"] == "invalid_request" diff --git a/tests/test_dual_solve.py b/tests/test_dual_solve.py index 12d7584d..0f0979fe 100644 --- a/tests/test_dual_solve.py +++ b/tests/test_dual_solve.py @@ -5,6 +5,7 @@ """ from __future__ import annotations +import json from pathlib import Path import pytest @@ -56,6 +57,32 @@ def test_changed_files_extracts_paths_from_git_diff(): assert ds.changed_files(diff) == ["README.md", "src/new.py"] +def test_recommendation_prefers_smaller_successful_diff(): + claude = ds.Candidate( + "claude", "b-claude", Path("/w/claude"), + diff="diff --git a/a.txt b/a.txt\n+one\n", ok=True, + ) + codex = ds.Candidate( + "codex", "b-codex", Path("/w/codex"), + diff="diff --git a/a.txt b/a.txt\n+one\n+two\n", ok=True, + ) + + rec = ds.recommendation([codex, claude]) + + assert rec["engine"] == "claude" + assert "smaller scoped diff" in (rec["reason"] or "") + + +def test_recommendation_avoids_tiebreaking_equal_scope(): + claude = ds.Candidate("claude", "b1", Path("/a"), diff="d", ok=True) + codex = ds.Candidate("codex", "b2", Path("/b"), diff="x", ok=True) + + rec = ds.recommendation([claude, codex]) + + assert rec["engine"] is None + assert "equally scoped" in (rec["reason"] or "") + + def test_require_engines_raises_when_missing(monkeypatch): monkeypatch.setattr(ds.shutil, "which", lambda b: None) with pytest.raises(RuntimeError, match="not on PATH"): @@ -140,6 +167,7 @@ def test_run_candidate_success_commits_and_captures_sha(tmp_path): assert cand.engine == "claude" assert cand.branch == "vouch-dual/3-fix-bug-claude" assert cand.diff == "patch text" and cand.sha == "abc123" + assert cand.log == "done" assert any(c[:5] == ["git", "-C", str(root), "worktree", "add"] for c in fr.calls) assert any(c[:4] == ["git", "-C", str(wt), "commit"] for c in fr.calls) assert any(c and c[0] == "claude" for c in fr.calls) @@ -390,8 +418,16 @@ def test_cli_dual_solve_json_is_noninteractive(monkeypatch, tmp_path): from vouch.cli import cli issue = ds.Issue("t", "b", number=1) - cands = [ds.Candidate("claude", "b1", tmp_path / "a", diff="DA", ok=True), - ds.Candidate("codex", "b2", tmp_path / "b", diff="DB", ok=True)] + cands = [ + ds.Candidate( + "claude", "b1", tmp_path / "a", + diff="diff --git a/a b/a\n+1\n", log="claude log", ok=True, + ), + ds.Candidate( + "codex", "b2", tmp_path / "b", + diff="diff --git a/b b/b\n+1\n+2\n", log="codex log", ok=True, + ), + ] monkeypatch.setattr("vouch.dual_solve._require_engines", lambda: None) monkeypatch.setattr("vouch.dual_solve.repo_root", lambda r, c: tmp_path) monkeypatch.setattr("vouch.dual_solve.prepare", @@ -403,7 +439,10 @@ def test_cli_dual_solve_json_is_noninteractive(monkeypatch, tmp_path): r = CliRunner().invoke(cli, ["dual-solve", "o/n#1", "--json"]) assert r.exit_code == 0, r.output - assert '"engine"' in r.output and "DA" in r.output and "DB" in r.output + body = json.loads(r.output) + assert body["recommendation"]["engine"] == "claude" + assert body["candidates"][0]["log"] == "claude log" + assert body["candidates"][1]["log"] == "codex log" # --json must not prompt and must not finalize/record. assert finalize_called["n"] == 0 diff --git a/tests/test_hooks.py b/tests/test_hooks.py new file mode 100644 index 00000000..9a096eff --- /dev/null +++ b/tests/test_hooks.py @@ -0,0 +1,85 @@ +"""The per-prompt hook injects relevant KB context with zero tool calls.""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from vouch import context, health, hooks +from vouch.models import Claim +from vouch.storage import KBStore + + +@pytest.fixture +def store(tmp_path: Path) -> KBStore: + s = KBStore.init(tmp_path) + src = s.put_source(b"e") + s.put_claim(Claim(id="c1", text="deploys run on tuesdays via ci", evidence=[src.id])) + health.rebuild_index(s) + return s + + +def _force_hit(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr( + context.index_db, "search_semantic", + lambda *a, **k: [("claim", "c1", "deploys run on tuesdays via ci", 0.9)], + ) + monkeypatch.setattr(context.index_db, "search", lambda *a, **k: []) + + +def test_empty_prompt_injects_nothing(store: KBStore) -> None: + assert hooks.build_claude_prompt_hook(store, json.dumps({"prompt": ""})) == "" + assert hooks.build_claude_prompt_hook(store, "") == "" + + +def test_relevant_prompt_yields_additional_context( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + _force_hit(monkeypatch) + out = hooks.build_claude_prompt_hook(store, json.dumps({"prompt": "when do deploys run"})) + env = json.loads(out) + assert env["hookSpecificOutput"]["hookEventName"] == "UserPromptSubmit" + assert "tuesdays" in env["hookSpecificOutput"]["additionalContext"] + + +def test_raw_non_json_stdin_is_tolerated( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + _force_hit(monkeypatch) + out = hooks.build_claude_prompt_hook(store, "when do deploys run") + assert "tuesdays" in json.loads(out)["hookSpecificOutput"]["additionalContext"] + + +def test_no_hits_injects_nothing( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setattr(context.index_db, "search_semantic", lambda *a, **k: []) + monkeypatch.setattr(context.index_db, "search", lambda *a, **k: []) + assert hooks.build_claude_prompt_hook(store, json.dumps({"prompt": "zzznomatch"})) == "" + + +def test_non_dict_json_payload_is_safe(store: KBStore) -> None: + for raw in ("null", "42", "true", "[1,2,3]", '"a string"'): + assert hooks.build_claude_prompt_hook(store, raw) == "" + + +def test_build_context_pack_exception_is_swallowed( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + def _boom(*a: object, **k: object) -> object: + raise RuntimeError("boom") + monkeypatch.setattr(hooks, "build_context_pack", _boom) + assert hooks.build_claude_prompt_hook(store, json.dumps({"prompt": "x"})) == "" + + +def test_context_hook_cli_always_exits_zero_without_kb( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + from click.testing import CliRunner + + from vouch.cli import cli + monkeypatch.chdir(tmp_path) # no .vouch here + result = CliRunner().invoke(cli, ["context-hook"], input='{"prompt":"anything"}') + assert result.exit_code == 0 diff --git a/tests/test_install_adapter.py b/tests/test_install_adapter.py index 83036e8c..64fe097f 100644 --- a/tests/test_install_adapter.py +++ b/tests/test_install_adapter.py @@ -319,6 +319,444 @@ def test_install_openclaw_is_idempotent(tmp_path: Path) -> None: } +# --- codex: T2 AGENTS.md fenced snippet (vouchdev/vouch#385) ---------------- + + +def test_codex_t2_appends_snippet_to_existing_agents_md(tmp_path: Path) -> None: + """Codex reads AGENTS.md for project instructions the way cursor does; + without the snippet a codex session gets the kb tools but no standing + guidance on recall-first or the review gate.""" + (tmp_path / "AGENTS.md").write_text("# My project\n\nExisting content.\n") + result = install("codex", target=tmp_path, tier="T2") + final = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + assert "Existing content." in final + assert "" in final + assert "" in final + assert "AGENTS.md" in result.appended + + +def test_codex_t2_creates_agents_md_when_absent(tmp_path: Path) -> None: + result = install("codex", target=tmp_path, tier="T2") + agents = tmp_path / "AGENTS.md" + assert agents.is_file() + assert "" in agents.read_text(encoding="utf-8") + assert "AGENTS.md" in result.written + + +def test_codex_t2_rerun_is_noop(tmp_path: Path) -> None: + install("codex", target=tmp_path, tier="T2") + before = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + again = install("codex", target=tmp_path, tier="T2") + after = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + assert before == after + assert "AGENTS.md" in again.skipped + assert "AGENTS.md" not in again.appended + + +def test_codex_t1_does_not_touch_agents_md(tmp_path: Path) -> None: + install("codex", target=tmp_path, tier="T1") + assert not (tmp_path / "AGENTS.md").exists() + + +def test_codex_snippet_stays_in_lockstep_with_cursor(tmp_path: Path) -> None: + """The two snippets carry the same invariants (recall first, all writes + via proposals, review stays human) and are phrased host-neutrally: the + only difference allowed is the host name itself.""" + codex = (ADAPTERS_DIR / "codex" / "AGENTS.md.snippet").read_text(encoding="utf-8") + cursor = (ADAPTERS_DIR / "cursor" / "AGENTS.md.snippet").read_text(encoding="utf-8") + assert codex == cursor.replace("cursor", "codex") + + +def test_fenced_refresh_replaces_edited_fence_body(tmp_path: Path) -> None: + """An edited fence body is brought back in sync within the markers; + user content outside the fence is untouched (vouchdev/vouch#385).""" + (tmp_path / "AGENTS.md").write_text("# Mine\n\nAbove.\n") + install("codex", target=tmp_path, tier="T2") + installed = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + + tampered = installed.replace( + "", + "\nstale hand edits\n", + ) + "\nBelow.\n" + (tmp_path / "AGENTS.md").write_text(tampered, encoding="utf-8") + + result = install("codex", target=tmp_path, tier="T2") + final = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + assert "stale hand edits" not in final + assert "Above." in final + assert "Below." in final + assert final.count("") == 1 + assert "AGENTS.md" in result.merged + assert "AGENTS.md" not in result.skipped + + +def test_fenced_append_when_marker_only_mentioned_in_prose(tmp_path: Path) -> None: + """A file that merely *mentions* the marker text (docs, a code sample) + has no standalone fence, so the snippet is appended rather than the + mention being mistaken for an existing install and skipped.""" + (tmp_path / "AGENTS.md").write_text( + "# Docs\n\nWe wrap vouch content in `` markers.\n", + encoding="utf-8", + ) + result = install("codex", target=tmp_path, tier="T2") + final = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + # the prose line survived, and a real standalone fence was appended + assert "We wrap vouch content in" in final + assert any(line.strip() == "" for line in final.splitlines()) + assert "AGENTS.md" in result.appended + assert "AGENTS.md" not in result.skipped + + +def test_fenced_refresh_ignores_marker_mention_below_real_fence(tmp_path: Path) -> None: + """Re-running stays a flat no-op even when the user pasted the marker + text into prose below the installed fence: the standalone fence is + up to date, the prose mention is not a second fence.""" + install("codex", target=tmp_path, tier="T2") + installed = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + (tmp_path / "AGENTS.md").write_text( + installed + "\nnote: the `` line is ours.\n", + encoding="utf-8", + ) + before = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + result = install("codex", target=tmp_path, tier="T2") + assert (tmp_path / "AGENTS.md").read_text(encoding="utf-8") == before + assert "AGENTS.md" in result.skipped + + +def test_fenced_refresh_leaves_unclosed_fence_alone(tmp_path: Path) -> None: + """A begin marker without an end marker is a corrupt state we refuse to + mangle — the file is left untouched and reported skipped.""" + (tmp_path / "AGENTS.md").write_text( + "content\n\nno end marker here\n", encoding="utf-8" + ) + before = (tmp_path / "AGENTS.md").read_text(encoding="utf-8") + result = install("codex", target=tmp_path, tier="T2") + assert (tmp_path / "AGENTS.md").read_text(encoding="utf-8") == before + assert "AGENTS.md" in result.skipped + + +# --- codex: T3 skills mirroring the vouch slash commands +# (vouchdev/vouch#386) ------------------------------------------------------ +# +# Scope decision from the ticket: codex custom prompts live only under +# ~/.codex/prompts/ (user-global) and are deprecated upstream in favour of +# skills, which DO have a project-local home at /.codex/skills/. +# Shipping skills keeps the #179 rule intact — a project-scoped install +# never touches home-directory state. + +_CODEX_SKILL_NAMES = ( + "vouch-ask", + "vouch-followup", + "vouch-propose-from-pr", + "vouch-recall", + "vouch-record", + "vouch-remember", + "vouch-resolve-issue", + "vouch-standup", + "vouch-status", +) + + +def _body_after_frontmatter(text: str) -> str: + parts = text.split("---", 2) + assert len(parts) == 3, "expected yaml frontmatter" + return parts[2].strip() + + +def test_install_codex_t3_ships_all_nine_skills(tmp_path: Path) -> None: + result = install("codex", target=tmp_path, tier="T3") + for name in _CODEX_SKILL_NAMES: + skill = tmp_path / ".codex" / "skills" / name / "SKILL.md" + assert skill.is_file(), f"missing {name}" + assert f".codex/skills/{name}/SKILL.md" in result.written + + +def test_codex_t3_writes_nothing_outside_the_project(tmp_path: Path) -> None: + """#179 invariant: every installed path stays under the target — a + project-scoped install must never reach ~/.codex.""" + result = install("codex", target=tmp_path, tier="T3") + for rel in (*result.written, *result.appended, *result.merged): + resolved = (tmp_path / rel).resolve() + assert resolved.is_relative_to(tmp_path.resolve()), rel + + +@pytest.mark.parametrize("name", _CODEX_SKILL_NAMES) +def test_codex_skills_stay_in_sync_with_claude_commands( + name: str, tmp_path: Path +) -> None: + """The skill bodies are the claude-code command bodies — referenced in + place from the openclaw mirror rather than forked, so one edit updates + every host and this test catches any drift.""" + install("codex", target=tmp_path, tier="T3") + skill = tmp_path / ".codex" / "skills" / name / "SKILL.md" + command = ( + REPO_ROOT / "adapters" / "claude-code" / ".claude" / "commands" / f"{name}.md" + ) + assert _body_after_frontmatter(skill.read_text(encoding="utf-8")) == ( + _body_after_frontmatter(command.read_text(encoding="utf-8")) + ), f"{name}: codex SKILL.md body drifted from the claude-code command" + + +def test_codex_skill_frontmatter_names_match_dirs(tmp_path: Path) -> None: + """Codex resolves a skill by its frontmatter name; a mismatch with the + directory name would ship a skill that answers to the wrong id.""" + install("codex", target=tmp_path, tier="T3") + for name in _CODEX_SKILL_NAMES: + text = (tmp_path / ".codex" / "skills" / name / "SKILL.md").read_text( + encoding="utf-8" + ) + frontmatter = text.split("---", 2)[1] + # Match the whole `name:` line, not a substring: `name: vouch-recall` + # must not be satisfied by `name: vouch-recall-typo`. + name_lines = [ + ln.strip() for ln in frontmatter.splitlines() + if ln.strip().startswith("name:") + ] + assert f"name: {name}" in name_lines, (name, name_lines) + + +def test_install_codex_t3_is_idempotent(tmp_path: Path) -> None: + install("codex", target=tmp_path, tier="T3") + second = install("codex", target=tmp_path, tier="T3") + assert second.written == [] + for name in _CODEX_SKILL_NAMES: + assert f".codex/skills/{name}/SKILL.md" in second.skipped + + +# --- codex: T4 hooks.json capture wiring (vouchdev/vouch#388) --------------- + + +def test_codex_t4_writes_hooks_json(tmp_path: Path) -> None: + """Fresh T4 install wires the Stop hook so a completed codex session + lands as a PENDING summary proposal with no manual steps.""" + result = install("codex", target=tmp_path, tier="T4") + hooks_path = tmp_path / ".codex" / "hooks.json" + assert hooks_path.is_file() + assert ".codex/hooks.json" in result.written + data = json.loads(hooks_path.read_text(encoding="utf-8")) + cmds = [ + h["command"] + for g in data["hooks"]["Stop"] + for h in g["hooks"] + ] + assert "vouch capture ingest-codex --hook" in cmds + + +def test_codex_t4_merges_into_existing_hooks_json(tmp_path: Path) -> None: + """An existing user hook is never silently overwritten — ours merges in + next to it via the json_merge machinery.""" + codex_dir = tmp_path / ".codex" + codex_dir.mkdir() + (codex_dir / "hooks.json").write_text(json.dumps({ + "hooks": { + "Stop": [ + {"hooks": [{"type": "command", "command": "my-own-stop-hook"}]} + ] + } + }), encoding="utf-8") + result = install("codex", target=tmp_path, tier="T4") + data = json.loads((codex_dir / "hooks.json").read_text(encoding="utf-8")) + cmds = [h["command"] for g in data["hooks"]["Stop"] for h in g["hooks"]] + assert "my-own-stop-hook" in cmds + assert "vouch capture ingest-codex --hook" in cmds + assert ".codex/hooks.json" in result.merged + + +def test_codex_t4_rerun_does_not_duplicate_hook(tmp_path: Path) -> None: + install("codex", target=tmp_path, tier="T4") + first = (tmp_path / ".codex" / "hooks.json").read_text(encoding="utf-8") + second = install("codex", target=tmp_path, tier="T4") + after = (tmp_path / ".codex" / "hooks.json").read_text(encoding="utf-8") + assert first == after + assert ".codex/hooks.json" in second.skipped + data = json.loads(after) + cmds = [h["command"] for g in data["hooks"]["Stop"] for h in g["hooks"]] + assert cmds.count("vouch capture ingest-codex --hook") == 1 + + +# --- codex: config.toml deep-merge (vouchdev/vouch#384) --------------------- + + +def test_codex_toml_merges_into_existing_config(tmp_path: Path) -> None: + """User already has .codex/config.toml — codex's *primary* config file. + The old plain-copy path silently skipped it, so vouch never got wired on + any project where codex was already configured. toml_merge adds + [mcp_servers.vouch] while preserving every unrelated table and value.""" + import tomllib + + codex_dir = tmp_path / ".codex" + codex_dir.mkdir() + (codex_dir / "config.toml").write_text( + 'model = "gpt-5"\napproval_policy = "never"\n\n' + '[mcp_servers.other]\ncommand = "other-server"\nargs = ["--fast"]\n', + encoding="utf-8", + ) + result = install("codex", target=tmp_path, tier="T1") + data = tomllib.loads((codex_dir / "config.toml").read_text(encoding="utf-8")) + + # user content preserved + assert data["model"] == "gpt-5" + assert data["approval_policy"] == "never" + assert data["mcp_servers"]["other"]["command"] == "other-server" + assert data["mcp_servers"]["other"]["args"] == ["--fast"] + + # vouch content merged in + assert data["mcp_servers"]["vouch"]["command"] == "vouch" + assert data["mcp_servers"]["vouch"]["args"] == ["serve"] + assert data["mcp_servers"]["vouch"]["env"]["VOUCH_AGENT"] == "codex" + + assert ".codex/config.toml" in result.merged + assert ".codex/config.toml" not in result.skipped + assert ".codex/config.toml" not in result.written + + +def test_codex_toml_merge_is_idempotent(tmp_path: Path) -> None: + codex_dir = tmp_path / ".codex" + codex_dir.mkdir() + (codex_dir / "config.toml").write_text('model = "gpt-5"\n', encoding="utf-8") + install("codex", target=tmp_path, tier="T1") + first = (codex_dir / "config.toml").read_text(encoding="utf-8") + second = install("codex", target=tmp_path, tier="T1") + after = (codex_dir / "config.toml").read_text(encoding="utf-8") + + assert first == after # no change on re-run + assert ".codex/config.toml" in second.skipped + assert ".codex/config.toml" not in second.merged + + +def test_codex_toml_fresh_install_writes_template(tmp_path: Path) -> None: + import tomllib + + result = install("codex", target=tmp_path, tier="T1") + cfg = tmp_path / ".codex" / "config.toml" + assert cfg.is_file() + data = tomllib.loads(cfg.read_text(encoding="utf-8")) + assert data["mcp_servers"]["vouch"]["command"] == "vouch" + assert ".codex/config.toml" in result.written + assert ".codex/config.toml" not in result.merged + + +def test_codex_toml_existing_vouch_entry_wins(tmp_path: Path) -> None: + """Conflict convention matches _install_json_merge: never clobber the + user. An existing [mcp_servers.vouch] value stays; only genuinely + missing keys (here the env table) are filled in.""" + import tomllib + + codex_dir = tmp_path / ".codex" + codex_dir.mkdir() + (codex_dir / "config.toml").write_text( + '[mcp_servers.vouch]\ncommand = "/opt/custom/vouch"\nargs = ["serve", "--debug"]\n', + encoding="utf-8", + ) + result = install("codex", target=tmp_path, tier="T1") + data = tomllib.loads((codex_dir / "config.toml").read_text(encoding="utf-8")) + + # the user's conflicting values win, deterministically + assert data["mcp_servers"]["vouch"]["command"] == "/opt/custom/vouch" + assert data["mcp_servers"]["vouch"]["args"] == ["serve", "--debug"] + # the missing env table is deep-merged in + assert data["mcp_servers"]["vouch"]["env"]["VOUCH_AGENT"] == "codex" + assert ".codex/config.toml" in result.merged + + +def test_codex_toml_malformed_existing_is_skipped(tmp_path: Path) -> None: + codex_dir = tmp_path / ".codex" + codex_dir.mkdir() + (codex_dir / "config.toml").write_text("= not valid toml [", encoding="utf-8") + before = (codex_dir / "config.toml").read_text(encoding="utf-8") + result = install("codex", target=tmp_path, tier="T1") + # unreadable user file is left untouched, not clobbered + assert (codex_dir / "config.toml").read_text(encoding="utf-8") == before + assert ".codex/config.toml" in result.skipped + assert ".codex/config.toml" not in result.merged + + +def test_toml_dumps_roundtrips_shipped_shapes() -> None: + """The hand-rolled serializer must faithfully re-emit everything tomllib + can hand it from the config shapes we merge into: nested tables, arrays, + inline tables inside arrays, quoted keys, and scalar types.""" + import tomllib + + from vouch.install_adapter import _toml_dumps + + data = { + "model": "gpt-5", + "temperature": 0.5, + "retries": 3, + "verbose": True, + "tags": ["a", "b"], + "weird key.name": "quoted", + "profiles": [{"name": "fast"}, {"name": "safe"}], + "mcp_servers": { + "vouch": { + "command": "vouch", + "args": ["serve"], + "env": {"VOUCH_AGENT": "codex"}, + }, + }, + } + assert tomllib.loads(_toml_dumps(data)) == data + + +def test_merge_toml_reports_no_change_when_subset() -> None: + from vouch.install_adapter import _merge_toml + + dst = {"a": {"b": 1, "c": [1, 2]}, "top": "x"} + src = {"a": {"b": 999}} # conflicting value: dst wins, nothing to add + assert _merge_toml(src, dst) is False + assert dst["a"]["b"] == 1 + + +def _write_manifest(tmp_path: Path, host: str, body: str, monkeypatch) -> None: + """Point the loader at a throwaway adapters dir holding one manifest.""" + import vouch.install_adapter as ia + + (tmp_path / host).mkdir(parents=True) + (tmp_path / host / "install.yaml").write_text(body, encoding="utf-8") + monkeypatch.setattr(ia, "ADAPTERS_DIR", tmp_path) + + +def test_manifest_non_boolean_flag_is_rejected(tmp_path: Path, monkeypatch) -> None: + """A quoted `"false"` is a non-empty string; bool() would read it as + True and silently enable a merge. The loader must reject it.""" + from vouch.install_adapter import _load_manifest + + _write_manifest(tmp_path, "badhost", ( + "host: badhost\n" + "tiers:\n" + " T1:\n" + ' - { src: a, dst: b, toml_merge: "false" }\n' + ), monkeypatch) + with pytest.raises(AdapterError, match="`toml_merge` must be a boolean"): + _load_manifest("badhost") + + +def test_manifest_multiple_strategies_rejected(tmp_path: Path, monkeypatch) -> None: + from vouch.install_adapter import _load_manifest + + _write_manifest(tmp_path, "badhost", ( + "host: badhost\n" + "tiers:\n" + " T1:\n" + " - { src: a, dst: b, json_merge: true, toml_merge: true }\n" + ), monkeypatch) + with pytest.raises(AdapterError, match="more than one of"): + _load_manifest("badhost") + + +def test_manifest_boolean_flags_still_accepted(tmp_path: Path, monkeypatch) -> None: + from vouch.install_adapter import _load_manifest + + _write_manifest(tmp_path, "okhost", ( + "host: okhost\n" + "tiers:\n" + " T1:\n" + " - { src: a, dst: b, toml_merge: true }\n" + ), monkeypatch) + manifest = _load_manifest("okhost") + assert manifest.tiers["T1"][0].toml_merge is True + + # --- error paths ---------------------------------------------------------- diff --git a/tests/test_jsonl_server.py b/tests/test_jsonl_server.py index 65627b76..dc1aa0a7 100644 --- a/tests/test_jsonl_server.py +++ b/tests/test_jsonl_server.py @@ -234,3 +234,17 @@ def test_jsonl_self_approval_allowed_with_trusted_agent_config( resp = handle_request({"id": "2", "method": "kb.approve", "params": {"proposal_id": pid}}) assert resp["ok"] + + +def test_jsonl_internal_error_omits_traceback(monkeypatch) -> None: + """HTTP /rpc clients must not receive server tracebacks on unexpected errors.""" + from vouch import jsonl_server + + def _boom(_params: dict) -> dict: + raise RuntimeError("secret internals") + + monkeypatch.setitem(jsonl_server.HANDLERS, "kb.status", _boom) + resp = handle_request({"id": "x", "method": "kb.status", "params": {}}) + assert resp["ok"] is False + assert resp["error"]["code"] == "internal_error" + assert "traceback" not in resp["error"] diff --git a/tests/test_mcp_profiles.py b/tests/test_mcp_profiles.py new file mode 100644 index 00000000..30bff123 --- /dev/null +++ b/tests/test_mcp_profiles.py @@ -0,0 +1,92 @@ +"""MCP tool profiles narrow the surface an agent sees (friendlier-mcp slice).""" + +from __future__ import annotations + +import pytest +from mcp.server.fastmcp import FastMCP + +from vouch import mcp_profiles +from vouch.capabilities import METHODS + +_MINIMAL_TOOLS = { + "kb_capabilities", "kb_status", "kb_context", "kb_search", + "kb_read_page", "kb_propose_claim", "kb_propose_page", "kb_list_pending", +} + + +def _make(name: str): + def fn(x: int = 0) -> int: + return x + fn.__name__ = name + return fn + + +def _fresh_mcp() -> FastMCP: + m = FastMCP("probe") + for method in METHODS: + m.tool()(_make("kb_" + method.split(".", 1)[1])) + return m + + +def test_full_profile_removes_nothing() -> None: + m = _fresh_mcp() + removed = mcp_profiles.apply_tool_profile(m, "full") + assert removed == [] + assert len(m._tool_manager._tools) == len(METHODS) + + +def test_minimal_exposes_core_only() -> None: + m = _fresh_mcp() + mcp_profiles.apply_tool_profile(m, "minimal") + assert set(m._tool_manager._tools) == _MINIMAL_TOOLS + + +def test_standard_is_superset_of_minimal() -> None: + assert mcp_profiles.PROFILES["minimal"] <= mcp_profiles.PROFILES["standard"] + + +def test_every_profile_is_subset_of_methods() -> None: + allm = set(METHODS) + for name, methods in mcp_profiles.PROFILES.items(): + assert methods <= allm, f"{name} references non-methods: {methods - allm}" + assert mcp_profiles.PROFILES["full"] == allm + + +def test_resolve_env_beats_config(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("VOUCH_TOOL_PROFILE", "full") + assert mcp_profiles.resolve_profile_name({"mcp": {"tool_profile": "minimal"}}) == "full" + + +def test_resolve_default_is_minimal(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.delenv("VOUCH_TOOL_PROFILE", raising=False) + assert mcp_profiles.resolve_profile_name(None) == "minimal" + + +def test_unknown_profile_falls_back_to_minimal(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("VOUCH_TOOL_PROFILE", "bogus") + assert mcp_profiles.resolve_profile_name(None) == "minimal" + + +def test_resolve_tolerates_non_dict_mcp_section(monkeypatch: pytest.MonkeyPatch) -> None: + """A bare `mcp:` key (YAML -> None) or non-dict mcp section must degrade + to the default, not crash `vouch serve`.""" + monkeypatch.delenv("VOUCH_TOOL_PROFILE", raising=False) + assert mcp_profiles.resolve_profile_name({"mcp": None}) == "minimal" + assert mcp_profiles.resolve_profile_name({"mcp": "oops"}) == "minimal" + assert mcp_profiles.resolve_profile_name({"mcp": {}}) == "minimal" + + +def test_compact_descriptions_trims_to_first_line() -> None: + m = FastMCP("probe") + + def kb_thing(x: int = 0) -> int: + """First line. + + Second paragraph with lots of detail the agent does not need. + """ + return x + + m.tool()(kb_thing) + changed = mcp_profiles.compact_descriptions(m) + assert changed == 1 + assert m._tool_manager._tools["kb_thing"].description == "First line." diff --git a/tests/test_openclaw_plugin_manifest.py b/tests/test_openclaw_plugin_manifest.py index d67b35bc..f111f726 100644 --- a/tests/test_openclaw_plugin_manifest.py +++ b/tests/test_openclaw_plugin_manifest.py @@ -138,13 +138,13 @@ def test_manifest_carries_no_dead_dialect_fields(manifest: dict) -> None: """The pre-2026.6 dialect's fields are silently ignored by the loader. Keeping them around would suggest they still do something; they don't. + Note: ``openclaw`` is now a live field carrying ``compat.pluginApi`` (#237). """ dead_fields = ( "family", "mcpServers", "shared_deps", "excluded_from_install", - "openclaw", "contracts", ) for dead in dead_fields: diff --git a/tests/test_retrieval_backend.py b/tests/test_retrieval_backend.py index 2dc30cd6..dcff5372 100644 --- a/tests/test_retrieval_backend.py +++ b/tests/test_retrieval_backend.py @@ -79,23 +79,101 @@ def test_backend_substring_only( assert _backends(pack) == {"substring"} -def test_backend_auto_prefers_embedding( +def test_backend_auto_now_fuses( store: KBStore, monkeypatch: pytest.MonkeyPatch ) -> None: - """Default `auto` tries embedding first when it returns hits.""" + """`auto` no longer waterfalls embedding-first; it fuses embedding + fts5 + (RRF) and tags hits `hybrid`.""" _force_semantic_hit(monkeypatch) _set_backend(store, "auto") pack = context.build_context_pack(store, query="JWT") - assert any(item["backend"] == "embedding" for item in pack["items"]) + assert pack["items"] + assert _backends(pack) == {"hybrid"} -def test_unset_backend_defaults_to_auto( +def test_unset_backend_fuses( store: KBStore, monkeypatch: pytest.MonkeyPatch ) -> None: - """A config with no retrieval.backend behaves like `auto`.""" + """A config with no retrieval.backend behaves like fused `auto`.""" _force_semantic_hit(monkeypatch) cfg = yaml.safe_load(store.config_path.read_text()) cfg.get("retrieval", {}).pop("backend", None) store.config_path.write_text(yaml.safe_dump(cfg)) pack = context.build_context_pack(store, query="JWT") - assert any(item["backend"] == "embedding" for item in pack["items"]) + assert _backends(pack) == {"hybrid"} + + +def test_backend_hybrid_merges_semantic_and_lexical( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + """`hybrid` returns the union of both retrievers, not first-non-empty.""" + src = store.put_source(b"e2") + store.put_claim(Claim(id="c2", text="OAuth refresh flow", evidence=[src.id])) + health.rebuild_index(store) + monkeypatch.setattr( + context.index_db, "search_semantic", + lambda *a, **k: [("claim", "c1", "JWT token rotation", 0.99)], + ) + monkeypatch.setattr( + context.index_db, "search", + lambda *a, **k: [("claim", "c2", "OAuth refresh flow", 0.88)], + ) + _set_backend(store, "hybrid") + pack = context.build_context_pack(store, query="auth") + assert {item["id"] for item in pack["items"]} == {"c1", "c2"} + assert _backends(pack) == {"hybrid"} + + +def test_near_duplicate_summaries_are_dropped( + store: KBStore, monkeypatch: pytest.MonkeyPatch +) -> None: + """An agent should not see the same fact twice.""" + src = store.put_source(b"z") + store.put_claim(Claim( + id="d1", text="the cache uses redis with a 60 second ttl", evidence=[src.id])) + store.put_claim(Claim( + id="d2", text="the cache uses redis with a 60 second ttl now", evidence=[src.id])) + health.rebuild_index(store) + monkeypatch.setattr( + context.index_db, "search_semantic", + lambda *a, **k: [ + ("claim", "d1", "the cache uses redis with a 60 second ttl", 0.90), + ("claim", "d2", "the cache uses redis with a 60 second ttl now", 0.89), + ], + ) + monkeypatch.setattr(context.index_db, "search", lambda *a, **k: []) + _set_backend(store, "hybrid") + pack = context.build_context_pack(store, query="cache") + assert {item["id"] for item in pack["items"]} == {"d1"} + + +def test_dedupe_keeps_highest_scored_regardless_of_input_order() -> None: + """Invariant: the highest-scored member of a near-duplicate cluster + survives even when items arrive out of score order (as graph-expansion + neighbours can).""" + from vouch.context import _dedupe_near_duplicates + from vouch.models import ContextItem + + lo = ContextItem(id="lo", type="claim", + summary="the cache uses redis with a 60 second ttl", + score=0.30, backend="hybrid", citations=[], freshness="unknown") + hi = ContextItem(id="hi", type="claim", + summary="the cache uses redis with a 60 second ttl now", + score=0.90, backend="hybrid", citations=[], freshness="unknown") + out = _dedupe_near_duplicates([lo, hi]) # deliberately low-score-first + assert [i.id for i in out] == ["hi"] + + +def test_dedupe_preserves_input_order_not_score_order() -> None: + """Survivors keep the caller's order (ranked hits first, appended + neighbours last) even when a later distinct item outscores an earlier one, + so budget eviction drops the tail, not the real matches.""" + from vouch.context import _dedupe_near_duplicates + from vouch.models import ContextItem + + a = ContextItem(id="a", type="claim", summary="alpha topic one", + score=0.02, backend="hybrid", citations=[], freshness="unknown") + b = ContextItem(id="b", type="claim", summary="beta subject two", + score=0.32, backend="graph", citations=[], freshness="unknown") + out = _dedupe_near_duplicates([a, b]) # distinct summaries, a first but lower-scored + assert [i.id for i in out] == ["a", "b"] diff --git a/tests/test_sessions.py b/tests/test_sessions.py index 9e3cbfed..e87cadb1 100644 --- a/tests/test_sessions.py +++ b/tests/test_sessions.py @@ -170,3 +170,37 @@ def test_crystallize_audit_event_records_summary_page_id( assert cryst_events, "no session.crystallize audit event found" last = cryst_events[-1] assert page_id in last["object_ids"], last["object_ids"] + + +def test_crystallize_retry_updates_existing_summary_page(store: KBStore) -> None: + from unittest.mock import patch + + src = store.put_source(b"e") + sess = sess_mod.session_start(store, agent="a", task="retry") + propose_claim(store, text="first", evidence=[src.id], proposed_by="a", session_id=sess.id) + propose_claim(store, text="second", evidence=[src.id], proposed_by="a", session_id=sess.id) + sess_mod.session_end(store, sess.id) + + real_approve = approve + calls = {"n": 0} + + def flaky_approve(*args, **kwargs): + calls["n"] += 1 + if calls["n"] == 2: + raise ValueError("transient") + return real_approve(*args, **kwargs) + + with patch("vouch.sessions.approve", side_effect=flaky_approve): + first = sess_mod.crystallize(store, sess.id, approver="u") + + assert len(first["approved"]) == 1 + assert len(first["failures"]) == 1 + assert first["summary_page_id"] is not None + + second = sess_mod.crystallize(store, sess.id, approver="u") + assert len(second["approved"]) == 1 + assert second["failures"] == [] + assert second["summary_page_id"] == first["summary_page_id"] + + summary = store.get_page(second["summary_page_id"]) + assert sorted(summary.claims) == sorted([c.id for c in store.list_claims()]) diff --git a/tests/test_storage.py b/tests/test_storage.py index 03ce0fba..7e0ccdd6 100644 --- a/tests/test_storage.py +++ b/tests/test_storage.py @@ -196,6 +196,47 @@ def test_update_claim_rejects_empty_evidence(store: KBStore) -> None: assert (store.kb_dir / "claims" / "c1.yaml").read_text() == persisted_before +@pytest.mark.parametrize("bad", ["", " ", "\n\t "]) +def test_claim_model_rejects_empty_text(store: KBStore, bad: str) -> None: + """Regression for #155: same shape as the #81 evidence validator, on the + text field. Empty / whitespace-only text is rejected at the model layer so + direct construction, store.put_claim, and bundle import all inherit it.""" + src = store.put_source(b"e") + with pytest.raises(ValidationError, match="text must not be empty"): + Claim(id="c1", text=bad, evidence=[src.id]) + + +def test_put_claim_rejects_empty_text(store: KBStore) -> None: + src = store.put_source(b"e") + with pytest.raises(ValidationError, match="text must not be empty"): + store.put_claim(Claim(id="c1", text=" ", evidence=[src.id])) + assert not (store.kb_dir / "claims" / "c1.yaml").exists() + + +def test_update_claim_rejects_empty_text(store: KBStore) -> None: + """A previously-populated claim cannot be mutated down to blank text and + silently re-persisted — update_claim re-validates via model_validate.""" + src = store.put_source(b"e") + store.put_claim(Claim(id="c1", text="cited", evidence=[src.id])) + persisted_before = (store.kb_dir / "claims" / "c1.yaml").read_text() + + c = store.get_claim("c1") + c.text = " " + + with pytest.raises(ValidationError, match="text must not be empty"): + store.update_claim(c) + assert (store.kb_dir / "claims" / "c1.yaml").read_text() == persisted_before + + +def test_claim_text_preserves_surrounding_whitespace_when_non_blank( + store: KBStore, +) -> None: + """The validator gates on emptiness only — it must not strip content.""" + src = store.put_source(b"e") + c = Claim(id="c1", text=" padded text ", evidence=[src.id]) + assert c.text == " padded text " + + # --- pages ---------------------------------------------------------------- @@ -217,6 +258,19 @@ def test_page_with_frontmatter_round_trip(store: KBStore) -> None: assert back.type == PageType.CONCEPT +@pytest.mark.parametrize("bad", ["", " ", "\n"]) +def test_page_model_rejects_empty_title(bad: str) -> None: + """Regression for #155 — empty / whitespace titles rejected on the model.""" + with pytest.raises(ValidationError, match="title must not be empty"): + Page(id="p1", title=bad) + + +def test_put_page_rejects_empty_title(store: KBStore) -> None: + with pytest.raises(ValidationError, match="title must not be empty"): + store.put_page(Page(id="p1", title=" ")) + assert not (store.kb_dir / "pages" / "p1.md").exists() + + # --- entities + relations ------------------------------------------------- @@ -226,6 +280,19 @@ def test_entity_round_trip(store: KBStore) -> None: assert back.name == "Foo" +@pytest.mark.parametrize("bad", ["", " ", "\t\n"]) +def test_entity_model_rejects_empty_name(bad: str) -> None: + """Regression for #155 — empty / whitespace names rejected on the model.""" + with pytest.raises(ValidationError, match="name must not be empty"): + Entity(id="e1", name=bad, type=EntityType.CONCEPT) + + +def test_put_entity_rejects_empty_name(store: KBStore) -> None: + with pytest.raises(ValidationError, match="name must not be empty"): + store.put_entity(Entity(id="e1", name=" ", type=EntityType.CONCEPT)) + assert not (store.kb_dir / "entities" / "e1.yaml").exists() + + def test_relation_round_trip(store: KBStore) -> None: store.put_entity(Entity(id="a", name="A", type=EntityType.PROJECT)) store.put_entity(Entity(id="b", name="B", type=EntityType.PROJECT)) diff --git a/tests/test_triage.py b/tests/test_triage.py new file mode 100644 index 00000000..ed46b8c9 --- /dev/null +++ b/tests/test_triage.py @@ -0,0 +1,433 @@ +"""Advisory triage scoring over the pending-review queue — issue #322.""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest +import yaml +from click.testing import CliRunner + +from vouch import triage +from vouch.cli import cli +from vouch.jsonl_server import HANDLERS, handle_request +from vouch.models import Claim, Entity, EntityType, Proposal, ProposalKind, ProposalStatus +from vouch.proposals import propose_claim, propose_entity +from vouch.storage import KBStore + +SIGNAL_NAMES = {"fit", "citation_quality", "duplication_risk", "contradiction_risk"} + + +@pytest.fixture +def store(tmp_path: Path) -> KBStore: + return KBStore.init(tmp_path) + + +def _enable_triage(store: KBStore, **overrides: object) -> None: + cfg = {"triage": {"enabled": True, **overrides}} + store.config_path.write_text(yaml.safe_dump(cfg), encoding="utf-8") + + +def _no_embedder(monkeypatch: pytest.MonkeyPatch) -> None: + def _raise(name: str | None = None) -> None: + raise KeyError("no embedder registered") + + monkeypatch.setattr("vouch.embeddings.get_embedder", _raise) + + +def _assert_block_shape(block: dict) -> None: + assert set(block) == {"recommendation", "score", "signals", "rationale"} + assert block["recommendation"] in {"approve", "reject", "needs-human"} + assert 0.0 <= block["score"] <= 1.0 + assert set(block["signals"]) == SIGNAL_NAMES + for sig in block["signals"].values(): + assert 0.0 <= sig["score"] <= 1.0 + assert isinstance(sig["reason"], str) and sig["reason"] + assert isinstance(block["rationale"], str) and block["rationale"] + + +# --- opt-in gate ------------------------------------------------------------- + + +def test_disabled_by_default_raises(store: KBStore) -> None: + with pytest.raises(triage.TriageError, match="disabled"): + triage.triage_pending(store) + + +def test_enabled_scores_pending_proposals(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + propose_claim(store, text="vouch requires citations", evidence=[src.id], proposed_by="agent") + results = triage.triage_pending(store) + assert len(results) == 1 + + +# --- output shape -------------------------------------------------------------- + + +def test_triage_block_shape(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + propose_claim(store, text="vouch requires citations", evidence=[src.id], proposed_by="agent") + [result] = triage.triage_pending(store) + assert result["kind"] == "claim" + _assert_block_shape(result["_meta"]["vouch_triage"]) + + +# --- no-write invariant --------------------------------------------------------- + + +def test_never_mutates_pending_queue(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + p1 = propose_claim(store, text="a claim", evidence=[src.id], proposed_by="agent").id + p2 = propose_entity(store, name="widget", entity_type="concept", proposed_by="agent").id + + before = {p.id for p in store.list_proposals(ProposalStatus.PENDING)} + triage.triage_pending(store) + after = {p.id for p in store.list_proposals(ProposalStatus.PENDING)} + + assert before == after == {p1, p2} + assert store.list_proposals(ProposalStatus.APPROVED) == [] + assert store.list_proposals(ProposalStatus.REJECTED) == [] + assert store.list_claims() == [] + assert store.list_entities() == [] + + +# --- citation_quality: reuses proposals._payload_block_reason ----------------- + + +def test_citation_quality_flags_dangling_ref_and_forces_reject( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + # Bypass propose_claim's own ref validation (store.put_proposal is raw + # I/O) to simulate a dangling reference slipping into the queue — + # the same shape proposals._payload_block_reason guards at approve time. + bad = Proposal( + id="bad-1", kind=ProposalKind.CLAIM, proposed_by="agent", + payload={ + "id": "c-bad", "text": "x", "type": "observation", "confidence": 0.7, + "evidence": ["missing-source"], "entities": [], "tags": [], + }, + ) + store.put_proposal(bad) + [result] = triage.triage_pending(store) + block = result["_meta"]["vouch_triage"] + assert block["signals"]["citation_quality"]["score"] == 0.0 + assert "missing-source" in block["signals"]["citation_quality"]["reason"] + assert block["recommendation"] == "reject" + + +def test_citation_quality_scores_clean_claim_positively( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + propose_claim(store, text="a well cited claim", evidence=[src.id], proposed_by="agent") + [result] = triage.triage_pending(store) + assert result["_meta"]["vouch_triage"]["signals"]["citation_quality"]["score"] > 0.0 + + +# --- duplication_risk: heuristic fallback (default in this dev env) ---------- + + +def test_duplication_risk_heuristic_fallback_flags_near_duplicate( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + text = "auth uses jwts in the authorization header for every request" + store.put_claim(Claim(id="c1", text=text, evidence=[src.id])) + propose_claim(store, text=text, evidence=[src.id], proposed_by="agent") + [result] = triage.triage_pending(store) + dup = result["_meta"]["vouch_triage"]["signals"]["duplication_risk"] + assert dup["score"] > 0.9 + assert "heuristic backend" in dup["reason"] + assert "c1" in dup["reason"] + + +def test_duplication_risk_heuristic_no_match_for_unrelated_text( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + store.put_claim(Claim(id="c1", text="apples and oranges", evidence=[src.id])) + propose_claim( + store, text="zebras run fast in the savanna", evidence=[src.id], proposed_by="agent", + ) + [result] = triage.triage_pending(store) + dup = result["_meta"]["vouch_triage"]["signals"]["duplication_risk"] + assert dup["score"] == 0.0 + assert "heuristic backend" in dup["reason"] + + +def test_duplication_risk_relation_exact_match( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + from vouch.models import Relation + + store.put_entity(Entity(id="a", name="A", type=EntityType.CONCEPT)) + store.put_entity(Entity(id="b", name="B", type=EntityType.CONCEPT)) + store.put_relation( + Relation(id="a--relates_to--b", source="a", relation="relates_to", target="b") + ) + dup_proposal = Proposal( + id="rel-1", kind=ProposalKind.RELATION, proposed_by="agent", + payload={ + "id": "a--relates_to--b-2", "source": "a", "relation": "relates_to", + "target": "b", "confidence": 0.7, "evidence": [], + }, + ) + store.put_proposal(dup_proposal) + [result] = triage.triage_pending(store) + dup = result["_meta"]["vouch_triage"]["signals"]["duplication_risk"] + assert dup["score"] == 1.0 + assert "already approved" in dup["reason"] + + +# --- fit: entity-overlap heuristic (no embeddings needed) --------------------- + + +def test_fit_scores_high_when_entities_already_known( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + store.put_entity(Entity(id="jwt", name="JWT", type=EntityType.CONCEPT)) + src = store.put_source(b"evidence") + propose_claim( + store, text="jwt tokens expire after an hour", evidence=[src.id], + entities=["jwt"], proposed_by="agent", + ) + [result] = triage.triage_pending(store) + fit = result["_meta"]["vouch_triage"]["signals"]["fit"] + assert fit["score"] == 1.0 + + +def test_fit_neutral_when_no_entities_referenced( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + propose_claim(store, text="an unrelated observation", evidence=[src.id], proposed_by="agent") + [result] = triage.triage_pending(store) + fit = result["_meta"]["vouch_triage"]["signals"]["fit"] + assert fit["score"] == 0.5 + + +# --- contradiction_risk -------------------------------------------------------- + + +def test_contradiction_risk_flags_polarity_conflict( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + store.put_entity(Entity(id="api", name="API", type=EntityType.CONCEPT)) + src = store.put_source(b"evidence") + store.put_claim(Claim( + id="c1", text="the api requires an auth token for every request", + evidence=[src.id], entities=["api"], + )) + propose_claim( + store, text="the api does not require an auth token for every request", + evidence=[src.id], entities=["api"], proposed_by="agent", + ) + [result] = triage.triage_pending(store) + conflict = result["_meta"]["vouch_triage"]["signals"]["contradiction_risk"] + assert conflict["score"] > 0.0 + assert "c1" in conflict["reason"] + + +def test_contradiction_risk_no_conflict_without_shared_entity( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + store.put_claim(Claim( + id="c1", text="the api requires an auth token for every request", + evidence=[src.id], + )) + propose_claim( + store, text="the api does not require an auth token for every request", + evidence=[src.id], proposed_by="agent", + ) + [result] = triage.triage_pending(store) + conflict = result["_meta"]["vouch_triage"]["signals"]["contradiction_risk"] + assert conflict["score"] == 0.0 + + +def test_contradiction_risk_not_applicable_to_non_claim_kind( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + propose_entity(store, name="widget", entity_type="concept", proposed_by="agent") + [result] = triage.triage_pending(store) + conflict = result["_meta"]["vouch_triage"]["signals"]["contradiction_risk"] + assert conflict["score"] == 0.0 + assert "only assessed for claim proposals" in conflict["reason"] + + +# --- embeddings-present path (requires numpy; skipped without it) ------------ + + +@pytest.fixture +def _mock_embedder() -> None: + pytest.importorskip("numpy") + from tests.embeddings._fakes import MockEmbedder + from vouch.embeddings import register + from vouch.embeddings.base import DEFAULT_MODEL_NAME + + register(DEFAULT_MODEL_NAME, lambda: MockEmbedder(dim=8)) + + +def test_duplication_risk_embedding_backend_flags_exact_duplicate( + store: KBStore, _mock_embedder: None, +) -> None: + _enable_triage(store) + src = store.put_source(b"evidence") + text = "auth uses jwts in the authorization header" + store.put_claim(Claim(id="c1", text=text, evidence=[src.id])) + propose_claim(store, text=text, evidence=[src.id], proposed_by="agent") + [result] = triage.triage_pending(store) + block = result["_meta"]["vouch_triage"] + dup = block["signals"]["duplication_risk"] + assert dup["score"] >= 0.95 + assert "embedding backend" in dup["reason"] + # A near-duplicate hit is penalized by duplication_risk and must not + # also inflate fit via the same signal (see _topical_fit_scores). + assert block["recommendation"] != "approve" + + +def test_backend_heuristic_config_forces_fallback_even_with_embedder( + store: KBStore, _mock_embedder: None, +) -> None: + _enable_triage(store, backend="heuristic") + src = store.put_source(b"evidence") + text = "auth uses jwts in the authorization header" + store.put_claim(Claim(id="c1", text=text, evidence=[src.id])) + propose_claim(store, text=text, evidence=[src.id], proposed_by="agent") + [result] = triage.triage_pending(store) + dup = result["_meta"]["vouch_triage"]["signals"]["duplication_risk"] + assert "heuristic backend" in dup["reason"] + + +# --- proposal_ids filter / config plumbing ------------------------------------ + + +def test_proposal_ids_filters_to_subset(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + p1 = propose_claim(store, text="first claim", evidence=[src.id], proposed_by="agent").id + propose_claim(store, text="second claim", evidence=[src.id], proposed_by="agent") + results = triage.triage_pending(store, proposal_ids=[p1]) + assert [r["id"] for r in results] == [p1] + + +def test_custom_weights_read_from_config(store: KBStore) -> None: + custom_weights = { + "fit": 1.0, "citation_quality": 0.0, "duplication_risk": 0.0, "contradiction_risk": 0.0, + } + _enable_triage(store, weights=custom_weights) + cfg = triage.triage_cfg(store) + assert cfg.weights == custom_weights + + +def test_disabled_config_value_keeps_default_false(store: KBStore) -> None: + raw = yaml.safe_dump({"triage": {"weights": {"fit": 0.9}}}) + store.config_path.write_text(raw, encoding="utf-8") + cfg = triage.triage_cfg(store) + assert cfg.enabled is False + + +# --- registration sites -------------------------------------------------------- + + +def test_jsonl_handler_registered() -> None: + assert "kb.triage_pending" in HANDLERS + + +def test_jsonl_triage_pending_disabled_returns_invalid_request( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + import vouch.jsonl_server as jsonl_server + + monkeypatch.setattr(jsonl_server, "_store", lambda: store) + resp = handle_request({"id": "1", "method": "kb.triage_pending", "params": {}}) + assert resp["ok"] is False + assert resp["error"]["code"] == "invalid_request" + + +def test_jsonl_triage_pending_enabled_returns_blocks( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + import vouch.jsonl_server as jsonl_server + + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + propose_claim(store, text="a claim", evidence=[src.id], proposed_by="agent") + monkeypatch.setattr(jsonl_server, "_store", lambda: store) + resp = handle_request({"id": "1", "method": "kb.triage_pending", "params": {}}) + assert resp["ok"] is True + [item] = resp["result"] + _assert_block_shape(item["_meta"]["vouch_triage"]) + + +def test_cli_triage_disabled_shows_clean_error( + store: KBStore, monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.chdir(store.root) + result = CliRunner().invoke(cli, ["triage"]) + assert result.exit_code != 0 + assert "Traceback" not in result.output + assert "Error:" in result.output + assert "disabled" in result.output + + +def test_cli_triage_json_output(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + propose_claim(store, text="a claim", evidence=[src.id], proposed_by="agent") + monkeypatch.chdir(store.root) + result = CliRunner().invoke(cli, ["triage", "--json"]) + assert result.exit_code == 0, result.output + data = json.loads(result.output) + _assert_block_shape(data[0]["_meta"]["vouch_triage"]) + + +def test_cli_triage_sorts_ranked_table(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: + _no_embedder(monkeypatch) + _enable_triage(store) + src = store.put_source(b"evidence") + propose_claim(store, text="a well cited unique claim", evidence=[src.id], proposed_by="agent") + bad = Proposal( + id="bad-1", kind=ProposalKind.CLAIM, proposed_by="agent", + payload={ + "id": "c-bad", "text": "y", "type": "observation", "confidence": 0.7, + "evidence": ["missing-source"], "entities": [], "tags": [], + }, + ) + store.put_proposal(bad) + monkeypatch.chdir(store.root) + result = CliRunner().invoke(cli, ["triage"]) + assert result.exit_code == 0, result.output + lines = [ln for ln in result.output.splitlines() if ln and ln[0].isdigit()] + scores = [float(ln.split()[0]) for ln in lines] + assert scores == sorted(scores, reverse=True) diff --git a/tests/test_web_dual_solve.py b/tests/test_web_dual_solve.py index 00a4bcd5..4ea528df 100644 --- a/tests/test_web_dual_solve.py +++ b/tests/test_web_dual_solve.py @@ -80,9 +80,11 @@ def __init__(self, *, repo_root, runner, image): def _fake_prepare(monkeypatch, *, calls): issue = ds.Issue("Fix bug", "body", number=4, url="u") cA = ds.Candidate("claude", "vouch-dual/4-fix-bug-claude", Path("/w/claude"), - diff="diff --git a/x b/x\n+1\n", sha="s1", ok=True) + diff="diff --git a/x b/x\n+1\n", sha="s1", + log="claude fixed it", ok=True) cX = ds.Candidate("codex", "vouch-dual/4-fix-bug-codex", Path("/w/codex"), - diff="diff --git a/y b/y\n+2\n", sha="s2", ok=True) + diff="diff --git a/y b/y\n+2\n+3\n", sha="s2", + log="codex fixed it too", ok=True) def fake(store, issue_ref, root, runner, *, claude_effort="high", codex_effort="high", autonomy="edit", dry_run=False, @@ -117,6 +119,9 @@ def test_run_starts_job_and_reaches_ready(git_kb, monkeypatch): assert [x["engine"] for x in state["candidates"]] == ["claude", "codex"] assert state["candidates"][0]["changed_files"] == ["x"] assert state["candidates"][1]["changed_files"] == ["y"] + assert state["candidates"][0]["log"] == "claude fixed it" + assert state["candidates"][1]["log"] == "codex fixed it too" + assert state["recommendation"]["engine"] == "claude" # autonomy is forced to edit regardless of input assert calls[0]["autonomy"] == "edit" @@ -200,6 +205,7 @@ def test_choose_winner_finalizes_and_returns_ids(git_kb, monkeypatch): assert r.json()["proposed_ids"] == ["prop-1", "prop-2"] assert r.json()["kept_branch"] == "vouch-dual/4-fix-bug-codex" assert r.json()["changed_files"] == ["y"] + assert r.json()["recommendation"]["engine"] == "claude" assert captured["winner"] == "codex" assert captured["record"] is True and captured["reason"] == "cleaner"