Skip to content

Implement the negative-fixture meta-check — prove every hard check bites (D-256…D-260) #286

Description

@StarshipSuperjam

The engine opened this item itself — you didn't create it.

What this is. Every hard check must be proven to bite — a standing, CI-enforced negative-fixture meta-check that runs each in-scope hard check-logic unit against a committed negative fixture (a seeded bad input) and asserts the expected hard finding. It is the up-front enforcing correlate to the weak after-the-fact "possibly inert" telemetry flag, and it closes the false-green-gate / checker-of-checkers risk: a hard check that is a green no-op reads to the operator as "verified" when nothing was actually checked.

  • Design of record: engine-planning decision log D-256 (authorize), resolved across D-257 (validation: the law + execution model), D-258 (check: the by-presence fixture grammar), D-259 (validators-core: the rule instance), D-260 (core: the dispatcher entry point + the core-kind fixtures).
  • Scope = logic-unit: each kind callable (the five closed core kinds + any module-added kind) carries at least one negative fixture; each custom/script instance carries its own; a data rule of a proven kind inherits the kind's proof.
  • Honest ceiling: it proves a witnessed negative trips the check today — never completeness against every input or stability under drift. The only carve-out is a unit with no statically-decidable CI failure path, which resolves to a disclosed not-applicable.

What happens next. Build it — no new check kind; it rides custom/script:

  • validators-core — the meta-check as a custom/script rule (the sibling of the first-run reference-closure check): self-covering (carries its own negative fixture; the guard must not be falsifiable by what it judges), failing closed on a present in-scope unit whose fixture is absent (rosters: kind callables from the validator's kind registry, custom/script instances from the check-rule directory listing); assert the finding by set-membership (id/severity present), never order/count.
  • core — a dispatcher entry point that runs one rule (or kind) against a caller-substituted target (extends the existing direct coherence-kind invocation; the target-substitution is the new affordance), and ship each of the five closed kinds' negative fixtures co-located with their callables.
  • fixtures namespace — a reserved, glob-excluded, coverage-exempt location under .engine/ (a Tier-2 leaf per the repository-topology two-tier model; non-surface like .engine/tools/ / .engine/boot/), so a committed bad input neither reds the real suite nor reads as an orphan surface.
  • custom/script contract — pin the fail-closed modes (missing script, nonzero exit, unreadable JSON output -> each a hard finding on finding.v1); the fixtures exercise each mode.
  • Backfill — author negative fixtures for the existing in-scope hard-check set (complete-final-state, not new-checks-only): the five closed core kinds, the reference-closure check (decidable legs only; its computed-path leg and no-op-after-retire state are disclosed not-applicable), PR-body completeness, the presence checks.
  • Operator-facing — validation's "What the operator is told": a passing check means the engine proved that gate catches a deliberately broken example — proof the gate works, not proof the change is correct (keep maintainer vocabulary out of operator strings).

Metadata

Metadata

Assignees

No one assigned

    Labels

    engineOpened by the engine about its own health (not your product).

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions