refactor(primeintellect): remove legacy single-turn adapter#372
Conversation
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — e6cbc516
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-15T05:16:22Z
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 0 (none) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 64.3s (2 bridge agents) |
| Total | 64.3s |
💰 Value — sound
Removes a PrimeIntellect Verifiers environment exporter that was added hours earlier, relocating runtime-execution surface to agent-runtime where it belongs per the repo's layering rule — clean, complete removal with a CHANGELOG migration pointer.
- What it does: Deletes src/primeintellect/index.ts (464 LOC) which exported RunRecords+scenarios into a PrimeIntellect Verifiers Python package (pyproject.toml, dataset.jsonl, run_records.jsonl, and a generated
load_environment()usingvf.SingleTurnEnvwith substring scoring). Also deletes its 146-line test, the./primeintellectpackage.json subpath entry, itsscripts/build-entries.mjsline, and the READ - Goals it achieves: 1) Respect the substrate/runtime layering rule that CLAUDE.md states explicitly —
agent-evalmust hold only primitives that 'make sense WITHOUT a running agent loop'. Generating and executing a Pythonvf.SingleTurnEnvis runtime-shaped, not substrate-shaped. 2) Avoid keeping a 'legacy single-turn adapter' whose substring-based reward (required_substrings→ fraction-of-hits) is a weaker evalu - Assessment: Good. The removal is complete and coherent: code, tests, package export, build entry, README section all removed; CHANGELOG marks it breaking with a concrete migration path; repo has zero remaining references to the removed symbol (only an unrelated
rendererslink in examples/publish-rl-dataset/README.md and a comment in src/rl/dataset.ts:247 remain, both pointing at a different PrimeIntellect p - Better / existing approach: none — this is the right approach. Searched
src/rl/(exporters.ts, run-record-adapters.ts, dataset.ts, corpus.ts) and the wider tree for any existing Verifiers-environment / SingleTurnEnv / substring-scoring capability — none exists; the RL module handles generic SFT/RL dataset export, not Python Verifiers packaging. The replacement lives in agent-runtime#553 per the PR body, which matches the l - Model: opencode/kimi-for-coding/k2p7
- Bridge attempts: 1
🎯 Usefulness — sound
Clean removal of a legacy single-turn PrimeIntellect export that used unsafe substring scoring, replaced by a full runtime-program adapter in the correct layer (agent-runtime); nothing in this repo still references it.
- Integration: Deletion is fully consistent — package.json subpath, build-entries.mjs entry, source file, test file, and README example all removed together; grep across *.ts and *.md finds zero remaining imports of the deleted module. The CHANGELOG (line 11) records the breaking change and points users to @tangle-network/agent-runtime/primeintellect. The replacement lives in a sibling package per tangle-network
- Fit with existing patterns: Matches the documented layering rule in CLAUDE.md: agent-eval is the substrate with zero upward deps on agent-runtime, and a Verifiers adapter that needs real runtime execution + full trace import belongs in agent-runtime, not here. Moving the real implementation to the runtime package and stripping the unsafe single-turn shim from the substrate is exactly the grain of this codebase.
- Real-world viability: The removed module scored via substring containment, which the CHANGELOG explicitly calls out as unsafe; the replacement does runtime execution with full trace import, so the deletion removes a known-weak scoring path rather than a working one. No error/edge-case concerns apply to a pure deletion with no remaining call sites.
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
No concerns — sound change, no better or existing approach found. ✅
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
✅ No Blockers —
|
| glm | kimi-code | aggregate | |
|---|---|---|---|
| Readiness | 95 | 92 | 92 |
| Confidence | 80 | 80 | 80 |
| Correctness | 95 | 92 | 92 |
| Security | 95 | 92 | 92 |
| Testing | 95 | 92 | 92 |
| Architecture | 95 | 92 | 92 |
Reviewer score is advisory once the run is complete and the verdict has no blockers.
Full multi-shot audit completed 4/4 planned shots over 4 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 4/4 planned shots over 4 changed files. Global verifier still owns final merge decision.
🟡 LOW Breaking subpath removal will hard-fail unmigrated consumers — package.json
Deleting the './primeintellect' export means any downstream consumer still importing '@tangle-network/agent-eval/primeintellect' gets ERR_PACKAGE_PATH_NOT_EXPORTED at import time (not a deprecation warning). This is intentional and documented as Breaking in CHANGELOG.md with a migration path to @tangle-network/agent-runtime/primeintellect, and acceptable under 0.x semver, but the blast radius depends on fleet consumers being migrated before this is published. No code change needed in this file; just confirm consumers are off the subpath before release.
tangletools · 2026-07-15T05:25:17Z · trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 1 non-blocking finding — e6cbc516
Full multi-shot audit completed 4/4 planned shots over 4 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 4/4 planned shots over 4 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-15T05:25:17Z · immutable trace
Summary
Checks
pnpm test: 3,217 passed, 2 skippedpnpm typecheckpnpm lint: exited 0 with 4 existing warnings outside this changepnpm buildpnpm verify:packagegit merge-tree --write-tree origin/main HEAD