PeasyAI: default to collecting-mode compilation in subprocess.run#969
PeasyAI: default to collecting-mode compilation in subprocess.run#969ankushdesai wants to merge 1 commit into
Conversation
PeasyAI's compile path shells out to `p compile` via three subprocess.run sites: CompilationService (the canonical entry, used by peasy-ai-compile and the fix-all loop), compile_utils.try_compile (legacy helper), and a candidate-scoring path in GenerationService. None of them previously propagated `P_COMPILER_COLLECT_ERRORS`. With P compiler 3.0+'s new multi-error mode (PRs #957/#963/#965), enabling the env var means the fix-all LLM loop converges in O(N/k) round trips instead of O(N) where k is the average errors-per-iteration — exactly the motivating use case for the whole multi-error initiative. ## Change At each subprocess.run site: ```python env = os.environ.copy() env.setdefault("P_COMPILER_COLLECT_ERRORS", "1") result = subprocess.run([...], ..., env=env) ``` `setdefault` (not direct assignment) means an explicit override in the parent shell still wins: - Unset → child sees "1" (collecting mode, the new default) - Set to "0" / "false" → child sees the user's value (strict mode) - Set to anything else truthy → child sees the user's value This is the right precedence: the user's explicit choice always overrides PeasyAI's preferred default. ## Sites changed - `Src/PeasyAI/src/core/services/compilation.py` (the canonical compile entry point; used by peasy-ai-compile MCP tool + the fix-iteratively loop) - `Src/PeasyAI/src/utils/compile_utils.py` (legacy helper used by evaluation pipelines) - `Src/PeasyAI/src/core/services/generation.py:1445` (candidate-scoring compile during code generation) `os` import added to `compilation.py`; the other two files already had it. ## Not changed - `Src/PeasyAI/src/utils/checker_utils.py` — these invoke `p check`, not `p compile`. The env var has no effect on `p check`. - `Src/PeasyAI/src/core/compilation/environment.py` — these are toolchain-validation calls (e.g. `dotnet --list-sdks`), not P compilation. - The error parser already supports multiple errors via `compilation.get_all_errors` and `error_parser.parse` (both iterate all matches with re.finditer). No parser changes needed. ## Follow-up (separate PR) Audit recommended updating `fix_iteratively` (`fixer.py`) to call `get_all_errors` instead of `parse_error` and send all errors to the LLM in one batch. That's a behavioral change to the fix loop; this PR is the prerequisite (env var must propagate first) but the actual batching work is its own PR with its own testing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Propagates P_COMPILER_COLLECT_ERRORS=1 (via os.environ.copy() + setdefault) to all three subprocess.run sites in PeasyAI that invoke p compile, so the P 3.0+ collecting-mode default lets the fix loop see all type errors at once instead of one per round trip. The default is overridable by the user's environment.
Changes:
- Default
P_COMPILER_COLLECT_ERRORS=1incompilation.py's compile entry point. - Same default propagated in legacy
compile_utils.try_compile. - Same default propagated in
generation._compile_check_candidates.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| Src/PeasyAI/src/core/services/compilation.py | Adds os import and passes an env with collecting-mode default to the canonical p compile subprocess call. |
| Src/PeasyAI/src/utils/compile_utils.py | Builds an env with collecting-mode default and forwards it to the evaluation-pipeline p compile invocation. |
| Src/PeasyAI/src/core/services/generation.py | Builds a subprocess_env with collecting-mode default for the candidate-scoring p compile call. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Superseded by #970, which makes multi-error mode the compiler default. PeasyAI no longer needs to propagate the env var because the compiler defaults to collecting mode for all callers. |
Behavioral change: `p compile` now defaults to multi-error mode (reports all type errors in one pass). Replaces the previous opt-in P_COMPILER_COLLECT_ERRORS environment variable with a proper CLI flag `--strict-errors` (`-se`) for opting back into legacy abort-on-first behavior. Supersedes #968 (docs about the env var) and #969 (PeasyAI subprocess env-var setup) — both now redundant. - `CompilerConfiguration` — both constructors default `ContinueOnError` to true. Removed the `ReadContinueOnErrorEnvVar` helper. - `ICompilerConfiguration.ContinueOnError` XML doc updated to describe the new default and the `--strict-errors` opt-out. - `PCompilerOptions` — new `--strict-errors` / `-se` boolean flag. When passed, the parser sets `ContinueOnError = false` AND reconstructs the `Diagnostics` collector + `Handler` so the strict-mode collector throws on first Report. Order matters: collector first, then handler (which holds a reference to it). The motivating use case (PeasyAI's `peasy-ai-fix-all` loop) needs collecting mode to converge in O(N/k) LLM round trips instead of O(N). Env var propagation across subprocess boundaries is fragile and discoverability is poor (users have to know to set it). Making it the default delivers the value to every user automatically. Risk of breakage: zero for valid programs (no behavior change), exit code unchanged for invalid programs (still 1). Only user-visible difference is the NUMBER of diagnostics emitted per failed compile — users now see all errors at once. CI scripts that grep for an exact error count may need updating; they can opt out via `--strict-errors`. - `DiagnosticCollectorTest` — removed `ContinueOnError_ReadsEnvVar` (env var is gone). Replaced with `CompilerConfiguration_DefaultsToCollectingMode` which verifies `ContinueOnError == true`, `Diagnostics.ContinueOnError == true`, and the `config.Diagnostics === config.Handler.Diagnostics` shared- instance invariant. - `Phase1DormancyTest` and `MultiErrorAcceptanceTest` are unaffected — both explicitly override `config.ContinueOnError` after construction to run each mode. - `StaticErrorValidator` only checks `exitCode == 1`, so all existing StaticError/* regression tests pass without modification. - `README.md` — new "Multi-Error Compilation (P 3.0+)" section in "What's New" with worked example showing default + opt-out - `ChangeList.md` — entry under "P 3.0 Changes" - `CLAUDE.md` — top-level "Multi-Error Type Checking (Compiler)" section for contributors; also added the `--strict-errors` example in the "Working with P Programs" snippet - MCP tool descriptions (`peasy-ai-compile`, `peasy-ai-fix-all`) — updated to reflect that multi-error mode is now default - Stale env-var comment in `MultipleErrors.p` updated to say "Default (collecting)" - Close PR #968 (env var docs, now obsolete) - Close PR #969 (PeasyAI env var propagation, now obsolete) - Phase 3 (#967) may need a small rebase since it modifies the env-var test in DiagnosticCollectorTest. The test is renamed in this PR, resolving cleanly: Phase 3's `[NonParallelizable]` attribute change becomes a no-op (the method it was attached to no longer exists). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
PeasyAI's compile path shells out to
p compilevia threesubprocess.runsites — none of which previously propagated theP_COMPILER_COLLECT_ERRORSenv var. With P compiler 3.0+'s new multi-error mode (#957/#963/#965), enabling the env var lets PeasyAI'speasy-ai-fix-allloop converge in O(N/k) LLM round trips instead of O(N) where k is the average errors-per-iteration. This is the motivating use case for the entire multi-error initiative.Change
At each of three subprocess.run sites:
setdefault(not assignment) preserves the user's explicit override:"1"(collecting mode, new default)"0"/"false"→ child sees the user's value (strict mode)Sites changed
Src/PeasyAI/src/core/services/compilation.py— canonical compile entry point used bypeasy-ai-compileMCP tool +fix_iterativelyloopSrc/PeasyAI/src/utils/compile_utils.py— legacy helper for evaluation pipelinesSrc/PeasyAI/src/core/services/generation.py:1445— candidate-scoring compile during code generationNot changed
Src/PeasyAI/src/utils/checker_utils.py— invokesp check, notp compile; env var has no effect thereSrc/PeasyAI/src/core/compilation/environment.py— toolchain validation calls (dotnet --list-sdksetc.), not P compilationcompilation.get_all_errorsanderror_parser.parse(both iterate all matches withre.finditer). No parser changes needed.Follow-up (separate PR)
The multi-agent audit recommended updating
fix_iterativelyto callget_all_errorsinstead ofparse_errorand send all errors to the LLM in one batch. That's a behavioral change to the fix loop. This PR is the prerequisite (env var must propagate first); the batching work is a separate PR.Test plan
Src/PeasyAI/tests/)peasy-ai-fix-allon a P file with 3 independent errors; observe that the LLM sees all 3 in one round trip🤖 Generated with Claude Code