P2: Audit representative public Python repositories

[tomdps]

Parent

Tracking parent #251; epic #243. This is one vertical delivery slice. Do not close or bypass the parent tracker from this child.

Goal

Test the exact packed artifact against pinned real repositories and preserve every supported, mismatched, and dynamic-unsupported result before authorizing copy.

Dependency contract

• P2: Add native OS, Python, and toolchain matrix evidence #273 (P2: Add native OS, Python, and toolchain matrix evidence)

Required behavior

1. Define a reviewed manifest of pinned public repo commits/licenses covering mypy/plugin/src-layout, Pyright execution environments, stub/namespace-heavy code, pytest/Ruff, and monorepo behavior; validate candidates at JIT time rather than fabricating profiles.
2. Separate dependency/setup acquisition from offline validation, then run source-checkout and exact packed-artifact paths over baseline plus seeded syntax/type/import/style/test/overlay/semantic ambiguity/rename-preview cases.
3. Compare compiler/checker/Ruff/pytest/editor-visible semantic behavior to normalized Opcore rows; preserve config/tool/timeout/crash/mismatch/cleanup/network mutation as non-pass and dynamic/reflection limits as explicit unsupported.
4. Record URL/commit/license/tree hash, setup ledger, tool versions, exact package/managed artifact hashes, before/after source hash, normalized results, source-vs-pack parity, and unsupported divergence list without committing upstream source.
5. Maintainer/subagent-owned. Do not launch Zeroshot unless the user explicitly re-delegates this slice after its JIT review.

Acceptance criteria

• At least one pinned profile for mypy/plugin/src, Pyright execution-environment, stubs/namespaces, and pytest/large-package behavior is reproducible.
• Claimed cells agree with external truth; mismatches block or narrow the cell instead of being normalized away.
• Source trees are unchanged after read-only paths; approved mutation cases are rolled back and cleanup/network ledgers pass.

Observable outcomes

• Clean success carries positive execution/provenance evidence; zero diagnostics alone is never proof.
• Findings and every unavailable/invalid/timeout/crash/resource/stale/unsupported state are machine-readable and fail or degrade exactly as the capability contract says.
• No implicit fallback, partial result, or skipped execution is reported as a pass.
• Target source, configuration, lockfiles, environments, and caches remain unchanged except where this issue explicitly owns a validation-gated atomic edit; every temporary resource/process is cleaned.

Non-goals

• benchmarking the ecosystem
• rewriting upstream config
• broad parity claims
• public docs changes

Verification

Run focused behavior first, then the configured repository proof only after the acceptance matrix works:

• npm run build
• the representative-repository audit runner and manifest validator
• source-vs-packed parity and offline/cleanup checks
• Attach exact real-tool/artifact versions, argv/config/cwd/source, normalized result examples, before/after cleanliness evidence, and packed-install proof where the slice changes public package behavior.

Execution rule

Implement only after a just-in-time review against current dev. Keep exactly one implementation run active. PR base is dev; require green checks, clean scoped diff, merged PR, closed child, parent checklist update, focused reproductions, and opcore-ci proof before starting the next child. Do not claim Python readiness/parity, ASP authority, old-tool replacement, security/SAST, all-stack support, automatic fixes, or a blended score.