Skip to content

Harden cumulative reference evidence contracts - #26

Merged
changeroa merged 36 commits into
codex/roadmap-06-promotion-governancefrom
codex/roadmap-07-final-hardening
Jul 22, 2026
Merged

Harden cumulative reference evidence contracts#26
changeroa merged 36 commits into
codex/roadmap-06-promotion-governancefrom
codex/roadmap-07-final-hardening

Conversation

@changeroa

@changeroa changeroa commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Purpose

Close the cumulative evidence, workflow, filesystem, parser, and operator-contract gaps found during full-stack review without rewriting the already-reviewed roadmap history.

What changed

  • Bind component-state evidence to a deterministic 33-file canonical-source digest and clean committed revision.
  • Require browser-authored, environment-scoped PNG identity and closed visual/DOM/AX/session manifests; reject substitution, replay, source drift, and schema smuggling.
  • Make module roots portable with fileURLToPath, including checkout paths containing spaces.
  • Harden token manifests/sources against duplicate JSON keys, symlinks, path escape, and forged artifacts.
  • Fail closed on unsupported workflow YAML notation, mutable/Docker action refs, checkout credential variants, and ambiguous ownership rows.
  • Preserve complete JSON CLI output through real Node stream pipes and reject ambiguous CLI argument ordering.
  • Clarify source/generated ownership and the component-evidence operator path.
  • Trust the exact checkout for session Git reads in Playwright job containers, regress the different-owner path, and recapture the source-bound component evidence without changing visual pixels.

Non-goals

No Layout/pattern/catalog changes, fifth domain, default profile, Storybook, required browser-matrix expansion, baseline update, real adopter/promotion claim, aggregate accessibility certification, product suitability, cross-browser guarantee, or owner approval.

Verification

  • Governance 144/144; consumer 86/86; consumer CLI 7/7.
  • Token adapters 42/42.
  • Component semantics 40/40; artifact/session 47/47; generator 13/13.
  • JSON direct/real-pipe integrity 4/4: complete 996-byte validator and 51,211-byte harness outputs twice.
  • Durable strict evidence: session 8e81d617-4087-4350-9100-6e03798d1ff8; source digest sha256:b3c88210f186b8c662666f73187dd7bd1cb02012079096516cb27a3142fd80bf; 42 files total; 40 runtime files; 30/30 channel passes; all ten visual PNG hashes unchanged.
  • Different-owner regression: source contract and all 46 artifact/session adversarial cases pass with GIT_TEST_ASSUME_DIFFERENT_OWNER=1; the exact safe.directory argv is asserted and no global Git config is written.
  • Exact HEAD d86a51396c186afe26a4b2fb4fdd51d0374af70c: review-work goal, QA, code, security, and context lanes all PASS/HIGH (security severity NONE); all changed MJS files are at most 250 physical lines; worktree clean.

Key commands:

node scripts/test-validate-governance.mjs --json
node scripts/test-validate-consumer-reference.mjs --json
node scripts/test-json-cli-output.mjs
node scripts/test-reference-adapters.mjs --json
npm run test:component-state
npm run validate:component-state

Durable evidence: /Users/victor/Documents/StyleGallery/.omo/evidence/consumer-reference-roadmap-implementation/pr7/upstream-ci-fix-recapture/ (kept outside commits).

Atomicity and rollback

This corrective layer contains 27 boundary-separated commits. Contract/code commits precede isolated evidence recaptures; later review findings each have a focused regression commit. The upstream-container fix is split into 017b96e (code), 17131d6 (evidence-only recapture), 11db754 (test-only review correction), 346344d (job-level pinned image identity), and d86a513 (review-driven test modularization). Roll back in reverse commit order from d86a513 through 36539df, or revert the complete 9965afa..d86a513 range as one PR layer. Do not cherry-pick evidence recaptures without their source-contract predecessors.

Stack

Base is codex/roadmap-06-promotion-governance; merge sequentially after roadmap 06. This PR does not merge or retarget earlier layers automatically.

@changeroa changeroa added the triage: changes-requested Review found blocking changes label Jul 15, 2026
@changeroa

Copy link
Copy Markdown
Owner Author

ULW triage — changes requested

Reviewed exact cumulative head d86a51396c186afe26a4b2fb4fdd51d0374af70c against base 9965afad8e526646267be894b3164b99be4efbf9 in isolated worktrees.

What passed: the complete root workflow; generated drift; all contract/harness jobs; JSON real-pipe 4/4; token adapter 42/42; promotion 74/74; component artifact/session 46+ cases; fresh Chromium component states 10/10 with 40 files; fresh sentinel plus negative controls; all four GitHub jobs. Path containment, strict JSON, source/session/scenario binding, action pins, checkout credentials, and ordinary cross-session/wrong-image substitution are materially improved.

The cumulative stack is still not safe to merge:

  1. Semantic profile safety still false-succeeds. reference-profile-contract.mjs:84-112 compares raw bytes. Render-identical profiles with different serialization pass; broken/empty foundation bindings pass; an unregistered malformed third governed profile is skipped by the hard-coded inventory; flattened adapter-name collisions are not rejected until build.
  2. Promotion policy blockers from Define shared reference promotion policy #25 are unchanged. Deferred RFCs without regression evidence, normative bypasses without claim-scoped evidence, failed claim evidence, and whitespace-only required fields return exit 0 / ok:true. The independent-consumer gateway remains structurally unreachable, and profile_record is schema-optional but runtime-mandatory. The relevant promotion files are byte-identical to Define shared reference promotion policy #25.
  3. Evidence provenance is not transitively closed. Baseline identity omits imported scripts/pattern-data.mjs; changing it changes renderer behavior while baseline and purity validators remain green. Component source identity also omits a loaded evidence-contract dependency. Calibration still binds only Node major 22.
  4. A non-image can pass as PNG evidence. A 24-byte signature/IHDR-only buffer with no CRC, IDAT, IEND, or pixels passed finalization and validation after coordinated expectation/sidecar binding; Chromium reports natural dimensions 0×0. Require real decode/chunk integrity and this negative case.
  5. The uploaded evidence root is not closed. Validation enumerates runtime/, but the workflow uploads the entire top-level root with if: always(). A rogue top-level file remains accepted and would be uploaded.
  6. Browser negative controls are not trustworthy/isolated. Fake Playwright output containing expected marker text is accepted on any nonzero exit, and the real negative harness leaves test-results files that make a subsequent OKF validation fail.
  7. The busy visual claim remains unproven. Fresh captures are visually clean, but disabled-busy surfaces show only disabled styling; the renderer has no busy cue, and removing busy yields identical pixels.

Additional inherited gaps include the null-manifest CLI crash, incomplete canonical manifest/schema parity, exact-revision equality, and same-page renderer reentrancy.

Outcome: triage: changes-requested. Do not merge #26 or any part of the #21#26 dependency stack until the cumulative endpoint closes these cases.

@changeroa
changeroa merged commit 9b36dbf into codex/roadmap-06-promotion-governance Jul 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage: changes-requested Review found blocking changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants