factgate

The deterministic verification gate for AI research agents.
A grounded-research skill whose every citation a machine can re-check — and a gate an LLM can't talk its way past.

---

An AI agent will happily tell you CVE-2026-99999 is critical, cite a version that was never published, or quote a page that doesn't say what it claims. factgate makes that impossible to ship silently: it runs a cheap agentic research pass, records every claim in a machine-checkable evidence ledger, and closes with a deterministic gate that re-hits the live sources.

• A fabricated CVE 404s on NVD and is deleted — not kept as "unverified."
• A quoted sentence must literally appear on the re-fetched page (substring, after normalization).
• A recommended pkg@x.y.z must be clean on OSV; a price row must carry a same-run fetch date.
• A row marked verified=yes that the gate cannot confirm is a FAILURE, not a warning.

The model does the judgment. The scripts do the fetching, caching, and checking. The gate has no LLM inside it, so a weaker model cannot flatter its way to a PASS.

The 10-second proof

node verify.mjs --demo --mode tech

This isn't a mock. It hits live NVD, OSV, npm, PyPI and arXiv right now:

real CVE-2026-44578 → found on NVD ✅        fake CVE-2026-99999 → 404, must be deleted ✅
ws@8.17.0 → VULNERABLE (3 advisories) ✅      ws@8.21.0 → clean on OSV ✅
fabricated quote → not on page, UNSUPPORTED ✅  verified=yes it can't confirm → FAIL ✅
✅ PASS

Every check is a real network round-trip to an authority. The CI badge above runs this same suite on Linux, Windows and macOS on every push — the proof can't silently rot.

Install

factgate is a cross-harness skill — it names capabilities, and thin adapters map them to your agent's native tools (Claude Code, Codex, Hermes, opencode, cursor, openclaw, gemini).

git clone https://github.com/agiwhitelist/factgate && cd factgate
node install.mjs              # auto-detects your harness; --harness all / --global also work

Zero install for the skill core: Node ≥ 20, standard library only — no npm install, no build. (Only the optional MCP server under mcp/ has dependencies, isolated there.)

How it works

Six auto-detected modes, each anchored to its own authoritative live source:

Mode	For	Grounded in
tech	pick/compare a library, framework, API	registries + NVD/OSV + arXiv/DOI
code	understand a codebase	the real repo (file:line + symbols must exist)
market	pricing / alternatives / competitors	live vendor pages, every price dated
debug	diagnose a failure	the actual error + stack-named files
fresh	what people say right now	dated posts on named platforms, inside a window
open	any other factual question	the cited page — re-fetched, quote must be on it

The pipeline is linear and cheap by default — one agent plus zero-token CLI tools:

prior → search → collect → ledger (pre-gate) → synthesize → verify (deterministic gate)
        ▲ scripts do all the fetching/caching/dedup      ▲ the model can't dodge this

collect.mjs fetches with a fallback ladder and a same-day disk cache; ledger.mjs bounces a paraphrased quote before the row is even added; verify.mjs re-fetches every live source at the end and exits 0=PASS · 1=FAIL · 2=usage · 3=UNVERIFIED. UNVERIFIED never prints the word PASS, so CI can't mistake "nothing was checkable" for success.

The output is a dated, cited report with an evidence ledger a future session — or CI, via reverify.mjs — can re-check when the facts move.

Why it exists

A stronger model needs less help not hallucinating a known-shaped fact — but no model knows what npm, NVD, a vendor, or Reddit published after its training cutoff. The durable value is freshness past the cutoff plus a re-verifiable artifact plus a gate no prompt can talk past. Every one of the skill's constraint rules is a real failure caught in review and turned into a mechanical check — including the bypasses the 6.1.0 adversarial panel found (planted-cache confirmation, fail-open on unreachable sources, all-short-quote evasion). See CHANGELOG.md.

Benchmarks

Honesty note: the cost structure (mechanics in zero-token scripts, model only judges) has one illustrative single-run datapoint in CHANGELOG.md — not a controlled A/B. A rigorous benchmark (multiple topics, symmetric arms, cost and accuracy per row, preregistered pass bar) is pending; this table will carry measured numbers or stay marked pending. No estimated numbers ship here.

Metric	Value
Gate self-check (13 live/structural demos, 3 OSes)	✅ passing in CI
Adversarial gate-bypasses found & closed (6.1.0 panel)	3 critical + 14 more
Cost multiplier vs legacy engine (controlled A/B)	⏳ pending
End-to-end research quality vs baseline (blind judge)	⏳ pending

Requirements

• Node ≥ 20 (uses built-in fetch, crypto, child_process — no packages).
• Network access at gate time (the gate's whole point is hitting live authorities). In a network-off sandbox the skill stops and says so — it never invents facts.
• Optional: a headless crawler (crwl) on PATH lets the gate read JS-walled pages; absent, those rows are reported unverifiable, never passed.

License

MIT — see LICENSE.