Implement the E002 offline instrumentation replayer#10
Conversation
There was a problem hiding this comment.
Pull request overview
Implements the E002 “offline instrumentation replayer” as a dependency-free, read-only Stage B pipeline that verifies the frozen E001 evidence archive, reconstructs tick-boundary state, emits deterministic exact-rational telemetry, and produces repeatable aggregate analysis and packaging.
Changes:
- Added
superloop_e002offline archive reader + replayer with deterministic telemetry emission, run summaries, and aggregate ROC-AUC/H4 analysis. - Added an evidence generator script and a protected GitHub Actions workflow to generate/repeat/package canonical E002 evidence under Python 3.13.
- Added/updated documentation and readiness/registry statuses to reflect E002 implementation availability and evidence-pending state.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/generate_e002_evidence.py | Generates E002 evidence, validates determinism by repeating replay, and packages output into a deterministic tarball + SHA256. |
| tests/test_e002_replayer.py | Adds coverage for archive verification, determinism, telemetry schema invariants, and “no E001 runtime imports” enforcement. |
| tests/README.md | Documents E002 test invariants and how to run the full implementation suite. |
| src/superloop_e002/replayer.py | Core E002 implementation: frozen-archive verification, state reconstruction, telemetry emission, and aggregate analysis. |
| src/superloop_e002/model.py | Canonical JSON/digest utilities, exact rational wire encoding, and exact ROC-AUC implementation. |
| src/superloop_e002/init.py | Exposes the public E002 API surface (EvidenceArchive, replay_*). |
| src/README.md | Documents E002 Stage B usage and canonical evidence generation command. |
| REPOSITORY_READINESS.md | Updates the critical path and marks E002 Stage B implementation checklist items as complete (evidence still pending). |
| registry/CONCEPT_REGISTRY.md | Updates E002 registry status to “Implemented / evidence pending”. |
| experiments/README.md | Updates E002 status and adds an implementation summary section. |
| experiments/E002/results/README.md | Documents the canonical generator + protected runner, and reiterates that no evidence is promoted yet. |
| .github/workflows/e002-canonical-evidence.yml | Adds a protected workflow to generate and upload digest-verified canonical evidence artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if output.exists() and any(output.iterdir()): | ||
| raise ValueError(f"output directory is not empty: {output}") |
| info = archive.gettarinfo(str(path), arcname=str(path.relative_to(source))) | ||
| info.mtime = 0 | ||
| info.uid = 0 | ||
| info.gid = 0 | ||
| info.uname = "" | ||
| info.gname = "" |
| def test_archive_and_all_source_digests_verify(self) -> None: | ||
| with EvidenceArchive(ARCHIVE) as archive: | ||
| self.assertEqual(72, len(archive.run_ids)) | ||
| self.assertEqual(ARCHIVE_SHA256, __import__("hashlib").sha256(ARCHIVE.read_bytes()).hexdigest()) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c995419ef2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| interlock_id = record["interlock_id"] | ||
| receiver_conductance = parse_rational(record["receiver_conductance"]) | ||
| candidate_flow = parse_rational(record["candidate_flow"]) | ||
| observations["T1"].append( |
There was a problem hiding this comment.
For the T1 comparison, the E002 spec defines the pneumatic score as 1 - F* only “for an interlock with ready demand” (experiments/E002/EXPERIMENT__E002__NONCAUSAL_PNEUMATIC_INSTRUMENTATION__v0.1__2026-07-14.md:269-273). This loop appends a T1 row for every interlock record, including records whose reconstructed source_ready is false; those no-demand negatives can dominate the AUC and H4 counts and change the experiment classification without evaluating the predeclared target population. Please skip T1 rows unless the interlock has ready demand/source readiness.
Useful? React with 👍 / 👎.
Summary
Boundaries
The implementation is read-only and does not import or invoke E001 transition code. It cannot authorize transfers, mutate capacity credit, reorder work, or promote the local dry-run into a canonical result. H1 is reported as
offline_supported_shadow_not_evaluated; optional shadow mode remains a later decision.Validation
PYTHONPATH=src python -m unittest discover -s tests -p "test_*.py" -v— 20 tests passedpython tools/check_repository.pygit diff --checkThe local run is implementation validation only. The canonical E002 evidence workflow will run from the merged Python 3.13 commit before results are interpreted or promoted.