Document the run-result format as a stable, machine-readable contract

[aslakhellesoy]

Motivation

Supervising agents and CI want to answer "which examples ran, which passed, what drifted, and where exactly did it fail?" without scraping test-runner text output. Varar already produces exactly this data — the per-spec run record (SpecResults, persisted as .var/<spec>.json) with span-anchored diffs (startOffset/endOffset, expected/actual per cell) and the drift baseline — but the format is currently described as "emerging" and consumed only by Varar's own editor/LSP/vitest plugin.

This matters for the AI-agent audience in particular: the current wave of spec-driven-development frameworks (GitHub Spec Kit, AWS Kiro, Tessl, Stravica RCF) all pitch "agent output checked against a human-owned contract", but their checking bottoms out in ID references and human review. Varar's run result is that contract, machine-checkable — it should be documented and consumable as such.

Proposal

• Add a reference page on the website documenting the run-result format: file location, top-level shape, the sourceHash fingerprint, example records, and the span-anchored diff structure (CellDiff, ReturnShapeError, drift entries).
• Declare a stability policy: version the format (or state the compatibility rule) so external consumers — supervising agents, CI gates, HTML overlays — can rely on it across Varar releases.
• Keep it cross-language: the record shape is already part of the cross-implementation consistency rules; the doc should be the language-neutral statement of it, alongside the conformance goldens.
• Optionally: a --json reporter (or documented pointer to the written files) so a consumer doesn't need to know the on-disk layout evolves.

Non-goals

• No new requirements-management layer or ID scheme — this only exposes what runs already produce.

References

• ADR 0002 (drift detection) — SpecResults, hashSource, run-result diagnostics
• CLAUDE.md "Return-based comparison" — "the emerging shared run-result format consumed by the editor, the LSP, and future HTML overlays"

[aslakhellesoy]

Another consumer audience for this, beyond agent supervision: compliance evidence / attestation pipelines.

The EU Cyber Resilience Act (reporting obligations from September 2026, full obligations December 2027) requires technical documentation demonstrating "effective and regular testing and review", and evidence-oriented SDLC tooling (e.g. Kosli, see this CRA article) works by attaching attestations — test results, scans — to an artifact's fingerprint.

A documented, stable run-result format is exactly what such a pipeline needs to attest "these documented behaviours were verified for this artifact" mechanically, rather than the weaker "the test suite passed":

• the run record ties verification to the spec source fingerprint (fnv1a:…), pinning which version of the documented behaviour was verified;
• span-anchored results make the evidence reviewable against the document itself;
• the drift baseline (varar.lock.json) is evidence that documented coverage did not silently decay over the support period — every acknowledgment is a reviewable commit.

No scope change needed — this is the same deliverable (reference page + stability policy + language-neutral shape). It just strengthens the case for the stability policy being explicit and versioned, since compliance consumers hold evidence for years, and gives the --json reporter idea a second concrete consumer.