P0 — Resolve Python projects, interpreters, environments, and tools canonically

[tomdps]

Parent

Parent EPIC: #243

Current dev baseline

Planned immediately before implementation against dev at c7f6a26a32f1b141b1dc8eac036a374a8f9cc9ea (PR #252 / #244 merged).

The current resolver is repository-root scoped. resolvePythonInterpreter({ repoRoot }) cannot accept a target file or distinguish nested projects. A reproduced repository with root Python 3.11 and services/api Python 3.12 selected the root .venv/bin/python, root cwd, and 3.11 for services/api/src/app.py; no file-aware project-context API exists. opcore init separately scans project/manager/env filenames, while status and validation each call the root-scoped toolchain resolver. This is observable decision drift, not only missing metadata.

Problem

Python tool discovery is POSIX-first and repository-root scoped. It recognizes a few config filenames, probes fixed .venv/venv paths or PATH, and does not establish nearest-project semantics. Nested monorepos, Windows executables, source roots, lockfiles, PDM/Pipenv/Poetry/uv, target version/platform, active environment precedence, and exact executable provenance can therefore be misreported or ignored. Status/init/install, checks, ASP, metrics, and installed execution can disagree about which Python project and toolchain own a file.

Decision

Add one canonical, read-only PythonProjectContext resolver in @the-open-engine/opcore-validation-python. Every dynamic Python consumer must use its result; static descriptors advertise the same contract/version and outcome vocabulary but must not fabricate repository-specific state.

The resolver contract belongs in @the-open-engine/opcore-contracts before product/provider packages consume it. Use schema id/version opcore.python.project-context.v1 and keep TypeScript types, runtime validators, packages/contracts/schemas/opcore-contracts.schema.json, package exports, packlists, and contract tests aligned.

Resolution input

Resolve one or more normalized repo-relative .py/.pyi targets against:

• canonical repository root;
• an injected read/list/exists/realpath workspace view so validation overlays and ASP host callbacks see the exact after-state without direct workspace reads by the provider;
• optional explicit interpreter/tool argv overrides;
• a sanitized environment projection and injected platform/process probe for deterministic POSIX/Windows tests.

Reject absolute/out-of-repo/parent-traversal target paths and ambiguous symlinked project/config paths. Reading config and probing --version/interpreter metadata is allowed; executing setup.py, imports, package code, manager install/sync hooks, or any source rewrite is forbidden.

Required output

For each target, return a deterministic context containing:

• normalized target, repository root, nearest project/package root, project boundary, source roots, namespace/flat/src/stub layout evidence, and the marker/config/lock files that caused the decision;
• requires-python, selected target Python version/platform/implementation when declared, and conflicts/unknowns without inventing an exact version;
• dependency/build manager evidence for plain pip/requirements, uv, Poetry, PDM, and Pipenv, including lockfile and static [build-system] metadata;
• selected interpreter as exact executable plus argv, cwd, source, version, implementation, platform, architecture and ABI/SOABI evidence;
• selected/configured mypy or Pyright, Ruff, pytest, and build environment as exact executable plus argv/cwd/source/config/version when available;
• a stable project key/context fingerprint derived only from normalized repo-relative identity plus relevant config/lock contents, explicit options, sanitized environment selection inputs, platform, and probe results;
• typed status/outcomes resolved, degraded, unsupported, or ambiguous, with machine-readable reasons for missing/invalid config, conflicting managers/targets, unavailable tools, timeout/signal/spawn/tool failure, unsupported target/platform, and path/symlink refusal.

Do not serialize secret-bearing environment values, full PATH contents, arbitrary subprocess output, or project source contents.

Precedence

Apply and test one explicit order:

1. nearest project boundary/marker for the target wins over repository-root defaults;
2. explicit Opcore argv override wins for that tool only;
3. active compatible environment for that project, then project-local environment (.venv, venv, env; POSIX bin/python*, Windows Scripts/python.exe/python.exe), then safe already-installed manager environment evidence, then PATH;
4. conflicting same-root manager locks/configs are ambiguous, not resolved by arbitrary brand preference;
5. repository tool config wins over defaults: Pyright pyrightconfig.json before pyproject.toml; Ruff ruff.toml, .ruff.toml, then pyproject.toml; mypy mypy.ini, .mypy.ini, pyproject.toml, setup.cfg, then tox.ini; pytest pytest.ini, pyproject.toml, tox.ini, then setup.cfg; build metadata comes from static pyproject.toml [build-system];
6. declared target constraints must be checked against the selected interpreter. A constraint is not rounded to an invented exact version.

Use no persistent cache. A process-local cache is optional and, if present, must key only the complete content/environment/probe fingerprint above and must never survive a process.

Implementation scope

Primary ownership:

• packages/contracts/ — shared context/result/reason/provenance contracts, schema, validators and tests;
• packages/validation-python/src/ — canonical resolver, static config parsing, path/layout/manager/tool/interpreter decisions, Node and injected-workspace adapters;
• packages/validation-policy/ and validation composition — inject/reuse the resolver for Python checks; group selected Python files by project context so syntax/type execution uses each owning project's interpreter/tool/cwd rather than one repo-root toolchain;
• packages/opcore/src/status-*, init.ts, reporting.ts, and advanced/product validation composition — remove duplicate Python project/env scans and surface/reuse the same dynamic contexts in status, scan/init/install preview and metric evidence;
• packages/asp-provider/ — resolve through host-backed file views, include the actual context fingerprint/outcome/tool provenance in assessment evidence/coverage, and keep all workspace content access on ASP callbacks;
• static Opcore managed-tool and ASP provider descriptors — advertise opcore.python.project-context.v1, supported outcomes, read-only/no-install behavior, and no repository-specific claims;
• fixtures/tests/package packlists/installed-bin coverage required by these changed public/package surfaces;
• AGENTS.md and mirrored CLAUDE.md — record the new canonical Python project-context ownership/convention because this adds a cross-package architectural pattern.

Delete or reduce the old root-scoped resolver and init.ts filename/env discovery so no second canonical decision path remains. Migrate every caller; do not leave compatibility aliases.

Acceptance criteria

1. Nearest-project correctness. A fixture monorepo containing independent root and nested Python projects with different requires-python, managers, configs and local interpreters resolves every .py/.pyi target to its nearest owning context. Syntax/type checks execute with that exact context. No file is validated with the root interpreter merely because the command was launched at repo root.
2. Layout matrix. Deterministic fixtures cover flat packages, src/ layouts, namespace packages, .pyi-only/stub packages, package __init__.py, and nested independent projects; source roots and package boundaries match the checked-in expected matrix.
3. Manager/config matrix. Checked-in expected resolver matrix covers plain requirements/pip, uv, Poetry, PDM and Pipenv; tool config precedence for mypy/Pyright, Ruff, pytest and build-system metadata; invalid config; and conflicting same-root manager/target evidence.
4. Environment/platform matrix. Injected deterministic probes cover explicit argv, active env, project-local env and PATH precedence on POSIX and Windows executable layouts, with exact argv/cwd/version/platform/architecture/ABI. No Windows host is claimed as exercised; matrix evidence is simulated unless a real runner exists.
5. Honest failures. Missing tools/configs, malformed probe output, non-zero exit, timeout, signal, spawn failure, incompatible target version, unsupported platform and ambiguous paths/managers produce typed visible outcomes. A clean zero-diagnostic pass is impossible when a required interpreter/tool/context is unresolved.
6. Read-only and safe. Resolver/status/init preview/check/ASP paths do not install/sync packages, execute project/setup/import code, write source/config/lock files, create environments/caches, or mutate the target repo. ASP project/config content is read only through host workspace callbacks. Symlink/path escape fixtures are refused visibly.
7. Cross-surface identity. For the same repo snapshot and target, status, scan/init/install preview, Python validation checks, ASP assessment evidence, metrics, and installed-artifact execution report/use the same project key, context fingerprint, outcome and selected executable/tool provenance. Static descriptors advertise only the matching contract/version/outcome vocabulary.
8. Overlay identity. A validation/ASP overlay that changes Python config or a target file is resolved from after-state content; the same tree materialized on disk yields the same context fingerprint/outcome. This is project-context parity only; full graph-overlay parity remains Hypothetical (validate pre-write) evaluation must be comprehensive — graph-backed checks are silently skipped #197.
9. Installed artifact. Clean packed-install tests exercise at least one nested multi-project fixture through public opcore status --json, opcore check ... --json, init/install preview, and opcore-asp-provider --stdio; source-checkout fallbacks, sibling paths and ambient workspace imports are excluded.
10. Matrix artifact. Attach or check in a machine-readable resolver matrix mapping fixture target to project root, manager/config evidence, source roots, interpreter/tool argv and expected outcome. PR completion evidence identifies exact OS/tool rows run versus simulated.
11. Existing contracts remain true. P0 — Make Python syntax validation compiler-truthful #244 compiler-truthful syntax fixtures and exact interpreter execution remain green, graph/edit/validation ownership boundaries remain intact, status/scan/check remain source-read-only, and no Python parity/readiness/Windows/all-stack/automatic-install claim is added.

Required verification

At minimum:

npm run build
node --test tests/contracts.test.mjs tests/validation-python.test.mjs tests/opcore-facade.test.mjs tests/asp-provider.test.mjs tests/installed-bins.test.mjs
npm run lint
bash ./scripts/ci/run-local-ci-equivalent.sh

Also run the checked-in resolver matrix command/test and attach its machine-readable output or artifact. The PR must state the actual interpreter/tool versions and real OS rows exercised.

Non-goals / forbidden scope

• Dependency solving, environment creation, package installation/sync, lockfile mutation, or manager-specific install hooks.
• Executing setup.py, imports, package code, arbitrary project commands, or automatic source rewrites.
• Python semantic inspect/edit, import/dead-code/relevant-test parity, Ruff/pytest execution, managed install recommendations, release readiness claims, receipt refreshes, or release version changes; those belong to ordered successors fix(python): share import resolution between validation and graph #209, Hypothetical (validate pre-write) evaluation must be comprehensive — graph-backed checks are silently skipped #197, P0 — Harden Python type checking and configuration semantics #245, fix(python): make ruff and pytest toolchain status honest #211, feat(inspect-edit): scope and isolate language-service projects #217, P1 — Add truthful Python LSP semantic inspection and navigation #248, P1 — Make Python rename and module moves semantic and atomic #249 and P2 — Ship honest Python readiness, setup, receipts, and cross-platform cutover #251.
• Persistent cache, daemon, background service, network access, third-party manager pinning, or claiming native Windows execution without a real runner.
• Duplicating P0 — Unify Python imports and all checks on one overlay graph snapshot #247 or P1 — Execute real Python relevant tests, lint, and format checks #250, changing public ASP host authority, claiming old-tool replacement, or touching generated native binaries/checksums and checked-in release receipts except where a failing package surface test proves a narrowly required fixture/packlist update.

Completion evidence

The PR must link the checked-in matrix artifact, the initial nested-project misselection reproduction, before/after public JSON for the same fixture, installed-artifact/ASP evidence, exact commands/results, real OS/interpreter/tool versions, proof that no source/config/lock/environment mutation occurred, and a scoped diff free of generated native/release churn.

[tomdps]

Completed in PR #254 and merged into dev as 91955e3193ceecc72c0e1f46ce21af79d834e5ed.

Final head 25084bd632eb657fec4101cf15a447f10f6aa15f passed required focused suites (177/177), the installed packed-bin suite (3/3), opcore check --changed --json, green GitHub CI/CodeQL/provenance, and the configured opcore-ci command proof (sha256:b8e6b9e171e929cf518c4f8768c1474ba46187462f7d9ea087a0812d5eac585d, exit 0, profile contract passed, no workspace drift). The checked-in matrix is packages/fixtures/validation-python/project-context/resolver-matrix.json; package/release artifact churn is absent from the scoped diff.