feat(conformance): exploratory parity discovery (spec 025) - #378
Merged
Conversation
Adds the hermetic/live module skeletons, the conformance/discovery/ data root, DiscoveryError/HarnessError variants, the discovery nextest profile with all-profile exclusions, and the foundational spine: deterministic PRNG (SplitMix64->xoshiro256**), grammar loader over the constraint inventory, normalized-diff-derived signature classifier, and the findings-queue model/loader/atomic-writer with D1/D5 validation. Wires the `discovery` command group (check/report/triage/split/scaffold) into the conformance CLI. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
Adds the MetamorphicRelation model, seven mandated mrl- relations each grounded in a resolvable clu-/bhv- id, V31 (missing/unresolvable ground) and V32 (mandated family with no record) validation, deacon-only relation evaluation (no oracle/Docker/network required), and the live discovery_metamorphic test binary selected only by [profile.discovery]. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
Registers discovery test binaries in the parity registry with a role-aware (live-campaign vs guard) reading, extends parity_registry_check with discovery-lane wiring + no-consumer-surface assertions, proves the hermetic discovery surface makes zero network calls, proves discovery report/check never gate on findings volume, and adds a nightly + workflow_dispatch discovery.yml CI lane. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
Adds grammar-driven constrained generation, the eleven-category mutation catalogue, the campaign driver (seed/tier/budget/timeout, admission cap, unsafe/unpinned-image guards), the differential comparison against the verified pinned oracle (reusing exec/oracle/ prereq/normalize, no new process-execution path), already-characterized suppression, and the discovery-campaign bin with --seed required and never defaulted. Verified against the live pinned oracle: a 44-candidate campaign reaches a 2.3% trivial-failure ratio, exercises all eleven mutation categories, and the admission cap correctly suppresses excess distinct signatures. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…rity-discovery Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…US1 merge US1 added a required `profile` field and a derived-id check to `Campaign` (data-model.md §1: seed ‖ pinnedInputSet ‖ lane ‖ profile ‖ tier). US3's synthetic_queue() test helper predated that change and built a campaign record with a hand-picked id and no profile, so it failed the strict loader post-merge. Compute the pinned input set and derive the campaign id the same way the real pipeline does instead of hand-picking either. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
Phase 6 of 025-exploratory-parity-discovery: place every finding in one of
six causes and collapse repeats, so the queue reflects distinct problems
rather than campaign volume.
Violation class D2 (T068) — classification arity and promotability, over
four shapes that are one defect: the queue asserting a judgement nobody
made, or holding one that cannot lead anywhere. "More than one
classification" is unrepresentable rather than unchecked (`Finding` carries
`Option<Classification>` and the strict loader rejects an array), so the
arity rule reduces to its only reachable violation.
State machine (T069) — `FindingState::may_transition_to` encodes exactly
data-model.md's five arrows, plus `Finding::{triage,promote,
mark_no_longer_reproducing}` and a typed `TransitionError`. Three
plausible-looking arrows are absent on purpose: `untriaged ->
no-longer-reproducing` (that state requires a classification, so the arrow
would manufacture a D2 out of a campaign merely not re-observing something
nobody had looked at), `no-longer-reproducing -> promoted` (promotion needs
current evidence), and `untriaged -> split` (a split separates a judgement,
so there must be one to find inadequate).
Split lineage (T070) — `split_finding` produces one child per witness, each
keyed by `parent || its witness ids`, because a child carries its parent's
signature verbatim and so cannot use the signature derivation. Deduplication
now refuses the whole lineage, including when the ancestor record is gone and
only children remain: without that clause the id lookup misses and a fresh
merged record resurrects under the ancestor's id, silently reverting the
reviewer's decision.
Admission cap (T071) — the cap was measured against every finding a campaign
touched, including ones the standing queue already carried and the run merely
re-witnessed. A queue grown past the cap would therefore reach it on
re-witnesses alone and freeze out every genuinely new signature forever. It
now counts only newly distinct signatures, per FR-034b.
CLI (T072) — `triage` and `split` delegate to the state machine rather than
re-deciding the lifecycle, and `split` authors the children it previously
deferred.
Report (T073) — adds the FR-031 grouping view (keyed by a reviewed behavior
where a promoted finding's case resolves, else by observable path; grouping
is a view, never a merge) and the queue-level suppression total, which is
what tells a reviewer the queue is a sample rather than everything.
Guard (T125) — no discovery source authors or extends an allowed-difference
entry. Reading stays permitted, since FR-017 requires it, and the guard
asserts the read still happens so a scan matching nothing cannot pass by
checking nothing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…ce (025 T096/T059)
Two deliberately-deferred integrations from US6 and US3, both blocked until now
only because each needed a file the other track had not yet created.
T096 — the `metamorphic` tier in the campaign driver
`campaign::run` now dispatches to a dedicated `run_metamorphic` BEFORE the
prerequisite step, because this tier has no prerequisite to acquire (research
D12): it compares deacon against deacon over a declared transformation, so there
is no oracle to verify, no Docker to probe, no network to reach. Previously the
tier reached the differential loop with `oracle == None` and failed as
`OracleUnverified` — the fail-loud guard doing its job on a tier that was never
routed anywhere.
The tier evaluates the committed `registry.metamorphic` catalogue with
`Sabotage::None` under the same wall-clock and per-candidate bounds the
differential uses; a relation that exceeds its bound is discarded AND counted,
never left to hang the run. Each violated relation's residual signatures are
admitted through the shared queue, keyed on `chan-structured-output` so a
metamorphic residual and a differential divergence at the same path deduplicate
against each other — they are the same defect observed two ways.
The plan IS the catalogue. `spaceCoveredFraction` is denominated by
`registry.metamorphic.len()`, not `planned_candidates`: there is nothing to
generate for this tier, so honoring the request's figure would report a
seven-relation run as 3.5% of a two-hundred-candidate plan it never had.
`mutationApplications` is all-zero with all eleven keys present (FR-010) and
`parseStageFailures` is 0 — the fixtures are authored, not drawn. An EMPTY
catalogue is an error, not a zero-relation run: V32 already forbids a mandated
family with no record, and a run over nothing reports exactly what a run in
which everything held reports.
Two limits are recorded in the code rather than left silent:
- `Characterization` (the FR-017 already-characterized suppression) is NOT wired
into this tier. That index is keyed to differential-style observable paths from
`cases/<area>.json` and the waivers, all of which describe a deacon-vs-reference
difference; a metamorphic residual is deacon-vs-deacon, so consulting it would
either suppress nothing or suppress by path collision. Left out visibly rather
than approximated.
- A violated SENSITIVITY relation yields no signature by construction (its failure
is the ABSENCE of a difference, and the catalogue deliberately refuses to key one
on the touched site, which would merge two unrelated defects). It cannot enter
the queue, so it is logged at `warn` naming the relation rather than dropped.
Extracted from the differential path, shared by both drivers rather than
duplicated: `AdmissionQueue` (dedup against the standing queue, the admission cap,
and FR-034b's "suppression is counted, never silent"), `pinned_input_set`, and
`complete` (exhaustion semantics, the append-only campaign history, the report).
US1's cap semantics are preserved verbatim — including that `admitted` counts
re-witnesses, so a campaign that re-witnesses `admission_cap` standing findings
has no budget left for a new one. That is a decision about the cap's meaning, not
a refactor, and is documented rather than changed.
T059 — evidence for the bin's exit-status contract
US1's `main()` already implemented the contract correctly; what was missing was
evidence. `discovery_campaign` now drives the COMPILED bin as a subprocess and
asserts all three statuses: `0` for a `--tier metamorphic --dry-run` run that
reached the whole catalogue, `2` for six malformed invocations, `1` for an
unverifiable oracle. T055 covers the library level; only a subprocess can observe
`main`'s `Result` -> `ExitCode` translation, which is where the contract is
actually enforced and where a doc comment is not evidence.
`env!("CARGO_BIN_EXE_discovery-campaign")` does not exist in this test — the bin
belongs to `parity-harness`, the test to `deacon` — so `prereq::cargo_bin`
generalizes `prereq::deacon_binary`'s build-and-read-cargo's-artifact rule instead
of guessing a `target/{debug,release}` path, which is the 023 T115 defect.
Verified: fmt --check clean; clippy --workspace --all-targets --all-features -D
warnings clean; `--profile default` 4233/4233; `--profile dev-fast` 3934/3934;
`--profile discovery` 24/24.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…to 025-exploratory-parity-discovery Resolves a conflict in campaign.rs between US4's admission-cap fix (measure the cap against newly-distinct signatures, not every id a campaign touched) and T096's AdmissionQueue extraction (which had preserved the pre-fix semantics verbatim, per its own report). Folds the newly_admitted fix into AdmissionQueue::offer so both changes are preserved: the cap logic now lives in one shared place and behaves correctly for both the differential and metamorphic tiers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
The merge conflict resolution moved finding_id into admitted.push() before its later use in newly_admitted.insert(), which doesn't compile. Clone at the first use site. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…s (025 US2) Implements Phase 4 (US2) of 025-exploratory-parity-discovery: reduce each finding's input while preserving its normalized signature, then hand the reviewer a complete, self-contained candidate. Reduction strategy (T047, hermetic, `deacon_conformance::discovery::shrink`) The seven-step catalogue of data-model.md §6 becomes executable, taking its reproduction predicate as a PARAMETER (`ReproductionProbe`) so the strategy is unit-testable against a synthetic predicate with no oracle, no Docker and no network (research D4/D5). The predicate has three answers, not a boolean: FR-023 needs the third, because a `bool` would discard a drifted signature at the moment it was observed. Termination is a property rather than a hope. Every accepted step strictly decreases a lexicographic `(mutations remaining, node count, non-minimal scalars)` triple; each component is reduced by exactly one step and increased by none that does not reduce an earlier one. Corruptions must lead, because `un-apply-mutation` can GROW the document — a size-only objective would let the shrinker oscillate between emptying and un-applying forever, spending the run's most expensive step on a ring it never leaves. `isMinimal` is true only when a complete pass over all seven steps accepted nothing, which is FR-021's claim exactly: minimal with respect to the DECLARED catalogue, finite and checkable — never the unfalsifiable claim that no smaller input exists. `first_further_reduction` re-establishes it independently, since a property that rests on how a loop exits is worth checking with a function whose only job is to look. `un-apply-mutation` is an exact reversal, not a category-directed guess: `mutate.rs` now records a `Reversal` per application, derived by diffing the operator's pre- and post-images at the document root so it cannot disagree with what the operator did. It is a LOCAL edit rather than a pre-image document — by the time reduction reaches it, earlier steps have already removed keys, and restoring a whole pre-image would silently undo all of them. No async runtime was added to `deacon-conformance`, deliberately. The first attempt took `tokio` as a dev-dependency to drive the async `reduce`, and `discovery_hermetic`'s SC-013 guard correctly rejected it: its claim is that the network capability is ABSENT, not merely unused, and a dev-dependency still puts a runtime in this crate's test binary. The tests drive the future with a five-line `block_on` over `Waker::noop()` instead — the synthetic predicate never suspends, so there is nothing to schedule. Live predicate (T048) and candidate assembly (T049–T051, `parity-harness`) `minimize::DifferentialProbe` re-runs `differential::compare` — no new execution path, no second normalization, no second opinion on what differs. Two things are held constant across every probe: the workspace shape (a probe tree differing from the candidate tree would measure the scaffold) and `deliberately_invalid` (it describes the candidate's provenance, and recomputing it as mutations are un-applied would move the tolerance boundary mid-reduction). `candidate::write` emits the six parts of data-model.md §9 under `target/discovery/candidates/<fnd-id>/`, with `raw.json` and `normalized.json` kept SEPARATE (FR-014): conflating them is what makes a `normalizer-defect` unreachable for a reviewer. `mapping.json` suggests a behavior only when a committed case already declares a document-shaped assertion on the same channel at the same observable path. FR-025 is held structurally — every proposal is filtered against `registry.behaviors`, so an id that does not resolve cannot escape the function. Ambiguity reports `match: none` with the candidates listed, because choosing one would be a coin flip wearing a suggestion's clothes. Campaign wiring (T052) Added to the existing differential loop; the tier dispatch and `AdmissionQueue` are unchanged apart from two read-only queries. Two gates on paying for a reduction — the signature must be new to this campaign, and the wall clock must remain — and neither may present the result as minimal: both route through `Reduction::not_attempted`, which carries `isMinimal: false` AND the reason (FR-022). Drifted signatures are admitted as separate findings whose witness names the REJECTED PROPOSAL they were seen on, not the reduction's own result, which does not reproduce them. A finding admitted from a drift gets no six-part candidate — it has no two-sided evidence set behind it — and the campaign reports which findings it did package rather than papering over the difference. Tests T041–T045 in `shrink.rs` (synthetic predicate): preservation, minimality (independently re-checked), determinism over the probe SEQUENCE and not merely the answer, budget exhaustion, and drift capture. T046/T124 live, verified against the pinned oracle 0.87.0. T046 goes past file existence: each candidate's fixture is copied to a fresh workspace and re-run against both implementations, and the claimed signature must still appear — the failure mode minimization introduces is precisely a fixture that no longer reproduces. T124 asserts the tier split as a PAIR, since `config-differential` running without Docker proves nothing unless `container-differential` refuses in the same environment. Verification: fmt clean, clippy `-D warnings` clean workspace-wide, `--profile discovery` 27/27 passed live, `--profile dev-fast` 3979/3979 passed. `--profile default` had two Docker timeouts (`test_alpine_container_starts`, `test_secret_masking_in_lifecycle_logs`) under 105 concurrent containers from other sessions; both pass in isolation in 13s and 57s. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
Phase 9 of 025-exploratory-parity-discovery: the real-world corpus becomes an ecological canary with recorded provenance, verified content, and a schedule of its own. T105/T106 — `CorpusEntry` + violation class D4, and the 33 pinned entries moved out of the Python fetcher into the Rust-owned strict-JSON `conformance/discovery/corpus.json`. D4 has two clauses answered in two places because they are answerable in two places: a non-immutable reference is a property of the manifest alone, so `corpus::check` rejects a branch/tag/HEAD/ `latest`/abbreviated SHA hermetically on every PR (research D8 — a validation that only runs when the network is up is a validation that does not run); a digest recorded and then removed is a property of a *change*, so `corpus::check_drift` answers it against an explicit baseline. `DiscoveryData` now loads all three collection files, so D4 runs wherever `discovery check` runs. T107 — `corpus_fetch`: a blob-filtered partial clone plus a sparse checkout, one round trip per entry, no API token and no rate limit (`microsoft/vscode` costs the same as `vscode-remote-try-node`). The digest is recorded at first materialization and verified thereafter; a mismatch is reported for that entry and the entry is NOT compared, because attributing a change in the upstream workspace to a difference between the implementations is a wrong conclusion arrived at confidently. A mismatch also never re-baselines the manifest — a self-healing check reports a problem once and then never again. `EntryStatus` has no "failed" catch-all: unreachable and digest-mismatch stay distinct facts. T108 — the `corpus` tier in the campaign driver, a third dispatch branch beside `run_metamorphic`, gated on the verified oracle AND the network (`HarnessError::NetworkUnavailable`). It reuses `differential::compare`, the tolerance index, `AdmissionQueue`, and `complete`, so FR-054's "same pipeline" holds by construction rather than by parallel implementation. Its plan is the manifest's own size; its witnesses name upstream provenance (repository, commit, path, verified digest) because corpus content is never vendored and a witness whose input nobody can retrieve names nothing. T128 — a weekly `schedule` for a dedicated `corpus` job in discovery.yml, separate from the nightly campaign. A canary that runs only on request cannot warn anyone; weekly rather than nightly because the corpus moves only when someone re-pins it (research D10). T109 — `fetch_realworld_corpus.py` is RETIRED. Two copies of one manifest drift, its documented workflow named runners 023 deleted, and the immutable-reference rule could not run from a Python tuple. `baseline.rs` now reads the entry names from the JSON manifest — same 33 names, same order, so the frozen baseline still regenerates byte-for-byte — and the surface is registered in `parity_registry_check`'s removed-surface guards. Tests: T099 (mutable reference is D4 with no network, SC-012), T100 (full provenance per entry), T104 (no corpus entry is a mutation seed, FR-008a/054a) in discovery_hermetic; T101 (digest mismatch), T102 (unreachable vs ran-and- found-nothing), T103 (pipeline parity + provenance + cross-campaign dedup) in discovery_campaign. Verified against the live pinned oracle and real network: 33/33 entries fetched and compared, 10 findings admitted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…ty-discovery Resolves conflicts in campaign.rs (CampaignRun struct-field union: US2's shrink_probes/candidates_root/candidates alongside US7's corpus_statuses, plus a merged module doc comment covering both minimization and the corpus tier) and in discovery_campaign.rs's test imports (union of both new imports). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…corpus tier The merge conflict resolution left run_corpus's Completion literal missing the candidates field US2 added, which doesn't compile. Corpus findings deliberately get no packaged reviewable candidate (Vec::new()): the existing candidate/minimize machinery was built around a single generated JSON document a campaign owns and can freely re-synthesize, but a corpus entry's workspace is a real third-party directory that may reference other files by relative path (Dockerfile, compose, local features) that this tier does not own and must not vendor (FR-053). Reducing just the JSON portion risks silently breaking those references. The witness still names full upstream provenance per FR-054; packaging a six-part candidate for real-world content is a separate design question, documented in place rather than papered over. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFsUBzDguiWtpaXXP74ogf
…25 US5) Make promotion of a discovery finding a human act with a stable behavior identity and a disposition, and make automatic promotion structurally impossible. - T079/T126 `discovery scaffold [--tolerate]`: emits skeleton behavior/ case/fixture records — or a scoped `wvr-` waiver plus its `allowedDifferences` entry — to **stdout only**, with `UNREVIEWED` sentinels the loader rejects. A blanket or unscoped tolerance is refused rather than emitted (FR-041). - T080 violation class **D3**: `promotedTo` must be present in state `promoted`, must resolve to a real registry case, and must be absent in every other state. - T081/T082 pipeline proof + `discovery-proof` bin: an injected difference must traverse generation → comparison → minimization → candidate → classification → promotable. Injection goes through the sealed `EvidenceSource` boundary, so perturbing an observer's return value does not compile (research D7). An injection that never lands is a fail-loud `inapplicable`, never "found nothing" (SC-016). - T074/T083 no-write-path guards: no discovery program can write a behavior, case, waiver, tolerated difference, disposition, or snapshot (SC-008), enforced both behaviourally and by source scan. - T075/T076/T077 promotion validation and certify isolation: a promoted finding validates as an ordinary case including `scenarioContext` (SC-009); a promotion missing an identity or disposition names what is missing (FR-038); `certify` is byte-identical with a queue full of unreviewed findings and with an empty one (SC-018). Verified: `cargo fmt --all -- --check`, `cargo clippy --all-targets --all-features -- -D warnings`, `discovery_hermetic` 27/27, `deacon-conformance` 431 unit + all integration, `parity-harness` 204 unit + all integration, `parity_registry_check` 15/15, `conformance validate` and `conformance discovery check` both clean, and `discovery-proof` traversing 2 injections with 0 failed / 0 inapplicable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bp2tFKG5y9cnSUHcsngi33
…thresholds Completes the final phase of 025-exploratory-parity-discovery (T110-T120, T129). No production code changed — this is documentation, measurement, and verification. Documentation: - CLAUDE.md gains an "Exploratory Parity Discovery" section (T110): the two sibling data roots, the hermetic/live split, the D1-D5 vs V-class distinction and their different consequences, and the never-gates rule. - CLAUDE.md's gate section becomes "Three lanes, two gates" with a summary table (T111), making explicit that discovery gates nothing. - conformance/RULES.md gains "Finding vs gap (do not conflate)" (T112): a finding is a candidate for an assertion backed by a reproducing witness and never blocks; a gap admits missing coverage and always blocks. The arrow between the roots points one way only. Quickstart drift corrected (T113): - The metamorphic example carried `"ground": "clu-7a2b91ce"`, an invented short id. Real `clu-` grounds are substance-anchored slugs, so copying the example produced a V31. Replaced with the committed record and a note showing a real clause id. - The corpus example's id and notes did not match the committed `images-python` entry; corrected, plus a note that corpus ids are derived, never hand-authored (a made-up one is D4). Measured thresholds (T114) — three real config-differential campaigns against the verified pinned oracle 0.87.0, recorded in research.md: - SC-002: worst observed parse-stage failure rate 2.50% against a 10% ceiling (seeds 0xa1b2c3d4 / 0x5eed0002 / 0x0badc0de). - SC-004: pooled median reduction 0.8438 over 13 independent minimizations, clearing the 0.80 floor. - Recorded why n is small: a candidate re-emitted for a signature the queue already carries deliberately does not re-minimize, so the metric must be recomputed from candidate provenance. Noted as a possible follow-up that the campaign outcome could carry the aggregate. SC-017 reviewer reproduction (T115) — reproduced a candidate from its fixture and named pins alone in 1s against a 600s budget. The one step that was not self-evident is now in the quickstart's Troubleshooting: the signature is defined on NORMALIZED output, and `drop_absent_optional` applies to deacon's `configuration` block only, so a fixture authoring an empty value for an ABSENT_OPTIONAL_KEYS key yields a `ref-only` signature with identical raw output on both sides. That is a `normalizer-defect`, and the underlying fix is `skip_serializing_if` on deacon (023 #T111). Verification (T116-T120, T129): - V30 clean: all 11 channels have a `reg-` record, no orphans, and every channel meets the SC-005 three-covering-case floor (minimum 3). - fmt + clippy --all-targets --all-features clean. - `make test-nextest` full gate: 4324 passed, 0 failed, 38 skipped. - Live discovery selection audit: zero live discovery tests in all six non-discovery profiles; the hermetic guard's 27 tests present in full/default/ci/dev-fast; the discovery profile selects exactly 30. - The whole discovery profile run green for the first time with the oracle installed: 18 campaign + 12 metamorphic tests. - certify's verdict is byte-identical with a 26-finding queue (including one claiming promotion into a nonexistent case) and with an empty one. - SC-015 budgets confirmed: 1800s wall clock, 60s hermetic and 300s container per-candidate, with timeouts discarded and counted into `candidatesDiscardedUnsafe` at all three sites rather than hanging. Campaign findings were deliberately NOT committed: the runs were for measurement, the seeds are recorded so they reproduce, and most of what they surfaced is the `drop_absent_optional` artifact above rather than a triage backlog worth carrying. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bp2tFKG5y9cnSUHcsngi33
github-actions
Bot
added
feature
The Windows `Test (MVP fast)` lane failed three tests introduced by this
branch. Two causes, both environmental — the production code is correct
on every platform.
**`conformance/discovery/**` was never pinned `-text`.** Every other
byte-compared artifact tree in this repository is (schemas, inventory,
spec prose, fixtures, snapshots, migration, obligations), and the
discovery data root has exactly the same discipline: the campaign driver
writes it through the canonical renderer, and both `discovery check` and
the hermetic guard byte-compare the committed files against a fresh
rendering. A Windows autocrlf checkout rewrote LF to CRLF, so
`raw.ends_with("}\n")` saw `}\r\n` and the comparison failed on bytes git
introduced — pointing the reader at a renderer that was working
correctly. Fixed by adding the pin, matching the seven that precede it.
Fixes:
- deacon::discovery_hermetic the_discovery_data_root_exists_and_is_canonically_rendered
- deacon-conformance discovery::queue::tests::the_rendered_empty_file_matches_the_committed_data_root
**`an_unreadable_file_is_not_mistaken_for_an_absent_one` simulates the
stat failure with `ENOTDIR`, which Windows does not have.** Windows maps
a stat under a non-directory to `ErrorKind::NotFound`, so the probe took
`load_file`'s legitimate "the file is absent" branch and `load` returned
`Ok` — the test could not reach the error path it exists to assert. The
guarantee itself is platform-independent (`load_file` treats only
`NotFound` as absent and turns every other stat error into a
`SchemaError`); it is the way of PROVOKING a non-`NotFound` stat error
that is Unix-specific. Gated `#[cfg(unix)]` with that reason, per the
convention CLAUDE.md records for genuinely Unix-only tests.
Verified: `git check-attr text` reports `unset` for all three data-root
files, the committed files carry zero CR bytes, fmt + clippy
--all-targets --all-features clean, discovery_hermetic 27/27 and the
deacon-conformance queue tests 50/50.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bp2tFKG5y9cnSUHcsngi33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
025-exploratory-parity-discoveryin full — all 129 tasks, zero deferred.The registry says whether each curated behavior is covered; the 024 coverage model says whether the scenario space is. This adds the third question: finding differences nobody curated, and handing each one over as a reviewable candidate that a human — never a program — may promote into the record.
What this adds
Two data roots, deliberately siblings.
conformance/discovery/(findings, campaigns, corpus) sits besideconformance/registry/, not inside it. No registry loader path reaches it, so nothing a campaign finds can influencevalidateorcertify. The only permitted cross-root reference points out of the queue (promotedTo→ a registry case, violation class D3).Never gates. A discovery command's exit status reflects whether it ran, never what it found. A campaign surfacing forty differences exits
0. The sole exception,discovery-proof, asserts a property of the machinery rather than a finding — so non-zero there means the pipeline is broken, which is exactly what should fail a lane. A stochastic gate would make green non-reproducible; that is why this lane can be scheduled in CI at all.Four campaign tiers:
metamorphic(deacon-only — no oracle, Docker, or network),config-differential(nightly),container-differential(invoked-only),corpus(weekly, network-backed).Promotion is a human act, structurally. There is no code path from a finding to a registry write — enforced behaviourally and by source scan.
discovery scaffoldprints to stdout withUNREVIEWEDsentinels the loader rejects.Violation classes D1–D5 over the discovery root, plus V31/V32 for the metamorphic relation catalogue.
Verification
make test-nextestfull gate: 4324 passed, 0 failed, 38 skippeddiscovery-proof: 2 injections traverse all 6 stages, 0 failed, 0 inapplicablefull/default/ci/dev-fast; thediscoveryprofile selects exactly 30. The profile filter is an explicit allow-list, never adiscovery_*glob, which would have silently droppeddiscovery_hermeticfrom the fast lane.certifybyte-identical with a 26-finding queue (one claiming promotion into a nonexistent case) and with an empty one--all-targets --all-featurescleanMeasured, not assumed
Three real campaigns against the verified oracle (seeds recorded in
research.mdso they reproduce):Notes for review
drop_absent_optionalartifact described below rather than a triage backlog worth carrying.ref-onlydifference while both CLIs produced identical raw output. Not a bug —drop_absent_optionalapplies to deacon'sconfigurationblock only, and cannot distinguish an authored"secrets": {}from an unset optional deacon serializes unconditionally. That is anormalizer-defect(a deliberate dead-end classification); the real fix isskip_serializing_ifon deacon, tracked as 023 #T111. The walkthrough is now in the quickstart's Troubleshooting.clu-id that would have produced a V31 if copied, and the corpus example's id did not match the committed entry.🤖 Generated with Claude Code
https://claude.ai/code/session_01Bp2tFKG5y9cnSUHcsngi33