feat(primeintellect): run runtime programs on Verifiers v1

[drewstone]

Summary

• generate a PrimeIntellect Verifiers v1 task package around any existing runtime program
• keep train and eval inputs disjoint and keep private answers outside the runner process
• support exact, reference-judge, and caller-command scoring without keyword heuristics
• import complete Prime traces into agent-eval RunRecords with strict token, cost, timing, and error validation
• publish the adapter as @tangle-network/agent-runtime/primeintellect with generated API docs

Checks

• pnpm verify:primeintellect: official PrimeIntellect-ai/verifiers@v0.2.0; exact scores [1, 0, 0], reference empty score 0, command reward 0.25, command metric 0.75
• pnpm test: 1,585 passed, 2 skipped
• pnpm typecheck
• pnpm lint: 447 files
• pnpm verify:package
• pnpm docs:check
• git merge-tree --write-tree origin/main HEAD

[tangletools]

✅ Auto-approved drewstone PR — 1a89652a

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-15T05:16:14Z}

[tangletools]

🟢 Value Audit — sound

Verdict	sound
Concerns	2 (1 low, 1 weak-concern)
Heuristic	0.0s
Duplication	0.0s
Interrogation	118.3s (2 bridge agents)
Total	118.3s

💰 Value — sound

Adds a PrimeIntellect Verifiers v1 bridge — generates a Python task package around any existing runtime program and imports Prime traces back into agent-eval RunRecords — built in the grain of the repo's export and substrate patterns; no existing equivalent found.

• What it does: Three capabilities under a new ./primeintellect subpath export: (1) createPrimeIntellectPackage/writePrimeIntellectPackage (src/primeintellect/package.ts:29) generate and atomically write a complete Python Verifiers v1 package (taskset.py, harness.py, pyproject, TOML configs, tasks.jsonl, hashed manifest) wrapping a caller's runner program, with validated disjoint train/eval splits and answe
• Goals it achieves: Connects the runtime to PrimeIntellect's RL/eval ecosystem so existing runtime programs (runPersonified/runAgentic/runLoop wrappers) can be trained and evaluated on Verifiers v1 without rewriting them in Python, while preserving the repo's core invariants: train/eval disjointness enforced at generation time (package.ts:146-151), private answers never entering the runner process (harness template w
• Assessment: Good on its merits. It follows the established subpath-export pattern (same registration points as candidate-execution/analyst-loop: package.json exports, scripts/verify-package-exports.mjs, scripts/gen-primitive-catalog.mjs, tsup, typedoc), reuses the substrate rather than reinventing it (createOpenAICompatibleBackend from src/backends.ts:208, RunRecord/validateRunRecord/canonicalJson fro
• Better / existing approach: None materially better. Searched for existing equivalents: no prior trace→RunRecord importer (grep toRunRecord|importTraces|traces.jsonl src/ returns nothing outside this change), no existing external-trainer/RL adapter (grep -ri primeintellect|verifiers hits only this PR's files plus generated docs), and the RL direction is documented but unimplemented until now. Alternatives considered: hand
• Model: opencode/kimi-for-coding/k2p7
• Bridge attempts: 1

🎯 Usefulness — sound

A well-scoped PrimeIntellect Verifiers v1 adapter that opens the runtime to an external RL/eval platform, wired exactly like the existing subpath exports (candidate-execution, mcp) with a real upstream integration test behind it.

• Integration: Reachable and correctly wired as a consumer-facing subpath. Registered identically to peer surfaces: package.json exports field (lines 67-71), tsup.config.ts entry (line 15), verify-package-exports.mjs import smoke test (lines 97-122), gen-primitive-catalog.mjs surface label (line 72), README consumer example (lines 120-177), and docs/api/primeintellect.md. Like ./candidate-execution, this is an a
• Fit with existing patterns: Strong fit, no competing pattern. It reuses the established primitives: createOpenAICompatibleBackend (src/backends.ts:208) via createPrimeIntellectBackend (runner.ts:51-62), with PrimeIntellectBackendOptions omitting apiKey/baseUrl/model so callers cannot override the intercepted endpoint; RunRecord + validateRunRecord from @tangle-network/agent-eval (traces.ts:1, 155) so Prime traces flow into t
• Real-world viability: Built for hostile inputs, not just the happy path. Private-field isolation is enforced on both ends: the generated harness (package.ts:476) only emits id/split/prompt/metadata/systemPrompt into TANGLE_PRIME_TASK_JSON, and the reader (runner.ts:95-99) rejects answer/reference/scoring/score if they ever leak back. Path-traversal is blocked for bundle files, scoring files, and runner files (assertRel
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

🔎 Heuristic Signals

🟡 Cruft: magic number added src/primeintellect/traces.ts

• return Math.max(0, (Math.max(...ends) - timing.start) * 1000)

💰 Value Audit

🟡 Chat-message validators duplicated between package.ts and runner.ts [duplication] ``

package.ts:318-431 and runner.ts:127-233 implement near-identical validation for system/user/assistant/tool messages, tool_calls, provider_state, and text/image_url content parts. Factor the shared validators into one module within src/primeintellect/ (e.g. messages.ts) and import from both — same behavior, one source of truth if the v1 message schema evolves.

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260715T052005Z}

[tangletools]

✅ Auto-approved drewstone PR — 06ae27da

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-15T05:25:08Z}

[tangletools]

🟢 Value Audit — sound

Verdict	sound
Concerns	2 (1 low, 1 weak-concern)
Heuristic	0.0s
Duplication	0.0s
Interrogation	169.8s (2 bridge agents)
Total	169.8s

💰 Value — sound

Adds a complete, well-validated PrimeIntellect Verifiers v1 adapter (package generator + rollout runner + trace→RunRecord importer) as a new /primeintellect subpath that reuses the existing backend and agent-eval primitives — a coherent new capability with no existing equivalent; ship.

• What it does: Introduces src/primeintellect/ (7 files, ~1,640 LOC) exporting three capabilities: (1) createPrimeIntellectPackage/writePrimeIntellectPackage generate a full PrimeIntellect Verifiers v1 Python package (pyproject, taskset.py, harness.py, tasks.jsonl, runner.json, sha256 manifest) from typed TS options, with strict validation — disjoint train/eval splits, duplicate public-input rejection, semv
• Goals it achieves: Let any existing agent-runtime program be trained and evaluated on PrimeIntellect's RL platform without forking the program: the same runner code runs inside Prime rollouts via the OpenAI-compatible endpoint, train/eval stay disjoint, answers never reach the runner process, and Prime traces flow back into the repo's existing agent-eval RunRecord analysis pipeline (reports, release checks). The sys
• Assessment: Good on its merits and in the grain of the codebase. The repo's established pattern is one subpath export per capability (/knowledge, /lifecycle, /mcp, /candidate-execution — see package.json exports and scripts/gen-primitive-catalog.mjs:72); this follows it exactly, including verify-package-exports coverage and generated API docs. It reuses existing primitives rather than reinventing: `cr
• Better / existing approach: none — this is the right approach. Searched for prior equivalents: no existing PrimeIntellect/verifiers code anywhere in history (git log --all | grep -i prime returns only this PR's two commits); src/candidate-execution/ is a local prepare/run/grade pipeline, not an external-platform adapter; src/environment-provider/ supplies sandbox backends, not eval-environment packaging; trace import r
• Model: opencode/kimi-for-coding/k2p7
• Bridge attempts: 1

🎯 Usefulness — sound

A well-factored adapter that wraps any Tangle runtime program as a PrimeIntellect Verifiers v1 RL/eval package and imports Prime's traces back into agent-eval RunRecords, reusing the right substrate primitives with a real end-to-end integration test against the upstream library.

• Integration: Fully reachable as the public subpath @tangle-network/agent-runtime/primeintellect (package.json:67-71, tsup.config.ts:15). It is wired into every gate the repo uses for any other subpath: verify-package-exports.mjs both lists it (line 52) and import-smoke-tests all eight public exports (lines 101-126); gen-primitive-catalog.mjs:73 labels it; docs/api/primeintellect.md + primitive-catalog.md:875
• Fit with existing patterns: Fits the codebase grain exactly. It follows the same subpath-export + index.ts-barrel-omitted shape as ./candidate-execution, ./mcp, ./profiles, ./knowledge. It reuses the canonical backend factory createOpenAICompatibleBackend (runner.ts:54) instead of forking transport logic, uses canonicalJson (package.ts:4) for input-dedup, and crucially projects Prime traces into the repo's common
• Real-world viability: Defensive well past the happy path. Package writes are atomic through a sibling temp dir with backup+restore on rename failure (package.ts:104-138); bundle paths are traversed-escape-checked (package.ts:278-290,108-113); writePrimeIntellectPackage refuses to clobber a non-generated directory by checking manifest.schema (package.ts:393-408) — covered by test at primeintellect.test.ts:153-159. S
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

🔎 Heuristic Signals

🟡 Cruft: magic number added src/primeintellect/traces.ts

• return Math.max(0, (Math.max(...ends) - timing.start) * 1000)

💰 Value Audit

🟡 Generated Python package lives as template strings inside package.ts [maintenance] ``

renderTaskset/renderHarness/renderPyproject embed the Prime-facing Python contract as string literals in src/primeintellect/package.ts, so upstream Verifiers API drift will surface only via the verify:primeintellect script (which requires uv + network and is not part of default pnpm test). Mitigation already present: the verifiers range is pinned (>=0.2.0,<0.3.0, package.ts:16) and the verify script asserts behavior against the real package. No better in-repo mechanism exists; noting only

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260715T052949Z}