feat(gate): insight quality gate — audit + forward enforcement#2
Merged
Conversation
Adds scripts/insight-gate.ts + scripts/lib/insight-gate.ts: a deterministic, recomputable semantic-quality audit over the insight corpus. Sibling to health.ts (structural validation) and verify-contracts.ts (doc/code surface) — this layer asks the quality questions those don't: is the stance directional? is it attributed to a real source? does its topic match its path? is it a near-duplicate? Checks (all deterministic, no LLM in the gate): - stance present / stance directional (heuristic floor) - attribution resolves to a known source (by normalized title or url) - topic matches file path - nearest-INSIGHT-neighbor cosine novelty: exact cosine on the stored float32 vectors (the vec0 table returns L2 distance, not cosine), INS-only via k=64 overfetch, with a block-threshold simulation and deduped triage pairs Phase 1 is audit-only: it reports, never blocks (exit 1 only on operational failure). brain.db is opened read-only. Outputs land in the gitignored knowledge-base/meta/. Forward enforcement (Phase 2) will reuse these checks. Tests: 47 (unit + in-memory sqlite-vec integration covering blob round-trip, PRI-/MM- filtering, and exact cosine). Run via: npm run gate -- --audit --all Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the audit checks into a forward-only enforcement gate. - enforceGate() (lib): partitions blocking failures from non-blocking warnings. Near-duplicate (>= cosine threshold, default 0.95) blocks when measurable; missing stance blocks by default. stance_directional (a heuristic) and attribution_resolves (would reject legitimate synthetic insights) are warnings by default, promotable via blockingChecks. An unembedded new insight WARNS (dup check unmeasurable) rather than silently skipping — fail-closing there would halt ingestion whenever Ollama is down. - insight-gate.ts --enforce: scopes via --changed (uncommitted insight files; fails CLOSED on git error and on changed files that don't load), --since, or all. Exits 1 on blocking failures. - post-ingest gate step: fatal, after embed (needs vectors) and before learn/auto-git (a blocked batch must not commit; placement preserves forward-only scoping since learn mutates existing insights). Mirrors the existing fatal reindex step. Conservative by design: blocks only on missing stance + near-identical duplicate, so it won't false-block legitimate or synthetic insights. Ratchet --max-similarity or promote warning-checks once trusted. Tests: 8 enforceGate cases. Verified e2e: --changed exit 0; blocking scope exit 1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gorajing
changed the title
The Phase 1/2 gate imports KB_ROOT from ./lib/kb-root, but the module was untracked (created by an in-progress refactor that was never committed), so the gate would not build on a fresh checkout. Track it here as the first committed code to depend on it. Self-contained (only imports node:path); resolves ZUHN_KB_ROOT or defaults to <repo>/knowledge-base. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A deterministic, recomputable semantic-quality gate for the insight corpus, in two commits.
Phase 1 — read-only audit (
093f344)scripts/lib/insight-gate.ts+scripts/insight-gate.ts: the layerhealth.ts(structural) andverify-contracts.ts(doc/code surface) don't cover.--auditnever exits 1 on quality; read-only DB; outputs to gitignoredmeta/.Phase 2 — forward enforcement (
67ffd2d)enforceGate(): blocking failures vs non-blocking warnings. Conservative default: blocks only on missing stance + near-duplicate (≥0.95).stance_directional(heuristic) andattribution(would reject synthetic insights) are warnings, promotable viablockingChecks. Unembedded new insight → warning, not silent skip.--enforce --changed: scopes to uncommitted insight files, fails closed on git error and on changed files that don't load.Gatestep inpost-ingest— after embed, before learn/auto-git (placement preserves forward-only scoping). Mirrors the fatal reindex step.Review
Test plan
--changedexit 0; blocking scope exit 1🤖 Generated with Claude Code