Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6419,7 +6419,7 @@ Improvement-loop coordinates. Present when called through improvementDriver.

###### costLedger?

`CostLedger`
`CostLedgerHandle`

Shared run-wide paid-call account supplied by agent-eval 0.117+.

Expand Down Expand Up @@ -13212,7 +13212,7 @@ Full-agentic `CandidateGenerator` (the `shots=N, sandbox=on` setting): run a rea

> **commandVerifier**(`command`, `args?`, `timeoutMs?`): [`Verifier`](#verifier)

Defined in: [improvement/agentic-generator.ts:860](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/agentic-generator.ts#L860)
Defined in: [improvement/agentic-generator.ts:863](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/agentic-generator.ts#L863)

A `Verifier` that runs a command in the worktree: exit 0 ⇒ ok, any other
exit ⇒ failed with stdout+stderr as feedback. The common case — verify by
Expand Down
24 changes: 17 additions & 7 deletions docs/api/primitive-catalog.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/canonical-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- This doc is the JUDGMENT layer: the mental model (§1), the AgentProfile law (§1.5), and the anti-reinvention decision table (§2) — WHICH primitive to reach for and what NOT to build. The export INVENTORY (WHAT exists) and per-symbol signatures + `file:line` are GENERATED into `docs/api/` (TypeDoc + `scripts/gen-primitive-catalog.mjs`, do NOT hand-edit) — that is the mechanical reference: `docs/api/primitive-catalog.md` is the never-stale list of every primitive to reuse. The freshness gate (`pnpm docs:freshness`) FAILS CI if a version pin, a cited `file:line`, a decision-table symbol, or the generated catalog drifts from source — see `docs/MAINTAINING.md`. Keep this file the small, hand-curated spine; never re-list the inventory here — point at the catalog. -->

> **Version 0.94.11.** The export inventory + per-symbol signatures live in the generated `docs/api/` reference: **`docs/api/primitive-catalog.md`** is the never-stale, grouped list of every primitive to reuse (own surface + the agent-eval judge / authenticity / verification / statistics / campaign / token-usage surfaces), with each one's import path and one-line summary read live from source; the per-module pages hold the full signatures. The pinned substrate is agent-eval `>=0.117.1 <0.118.0`; the sandbox substrate that materializes profiles into harness shapes is `@tangle-network/sandbox` (peer `>=0.8.0 <1.0.0`). The neutral contract types (`AgentProfile`, `AgentProfileMcpServer`, `HarnessType`, `ReasoningEffort`, `Part`/`ToolPart`/`ToolState`, plus environment-provider types) are owned by **`@tangle-network/agent-interface`** (peer `>=0.25.0 <0.26.0`) — the single source of truth. Substrate primitives are re-exported through `@tangle-network/agent-eval/contract` (or `/campaign`), not local to this package — the catalog's §2 shows exactly which subpath each lives under.
> **Version 0.94.12.** The export inventory + per-symbol signatures live in the generated `docs/api/` reference: **`docs/api/primitive-catalog.md`** is the never-stale, grouped list of every primitive to reuse (own surface + the agent-eval judge / authenticity / verification / statistics / campaign / token-usage surfaces), with each one's import path and one-line summary read live from source; the per-module pages hold the full signatures. The pinned substrate is agent-eval `>=0.120.1 <0.121.0`; the sandbox substrate that materializes profiles into harness shapes is `@tangle-network/sandbox` (peer `>=0.8.0 <1.0.0`). The neutral contract types (`AgentProfile`, `AgentProfileMcpServer`, `HarnessType`, `ReasoningEffort`, `Part`/`ToolPart`/`ToolState`, plus environment-provider types) are owned by **`@tangle-network/agent-interface`** (peer `>=0.25.0 <0.26.0`) — the single source of truth. Substrate primitives are re-exported through `@tangle-network/agent-eval/contract` (or `/campaign`), not local to this package — the catalog's §2 shows exactly which subpath each lives under.
>
> **`./loops` is the runtime barrel** — `package.json` maps it to `src/runtime/index.ts`. Everything below labelled `/loops` is the recursive-atom + loop-kernel surface.
>
Expand Down
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-runtime",
"version": "0.94.11",
"version": "0.94.12",
"description": "Shared task-lifecycle skeleton for agents: a recursive loop kernel for chat turns, one-shot tasks, and multi-attempt loops, with trace capture and eval-gated self-improvement. Domain behavior lives in adapters; scoring and ship-gates in @tangle-network/agent-eval.",
"homepage": "https://github.com/tangle-network/agent-runtime#readme",
"repository": {
Expand Down Expand Up @@ -116,7 +116,7 @@
},
"devDependencies": {
"@biomejs/biome": "^2.4.15",
"@tangle-network/agent-eval": "0.117.1",
"@tangle-network/agent-eval": "0.120.1",
"@tangle-network/agent-interface": "^0.25.0",
"@tangle-network/sandbox": "^0.9.7",
"@types/node": "^25.9.3",
Expand Down Expand Up @@ -147,7 +147,7 @@
"license": "MIT",
"packageManager": "pnpm@10.28.0",
"peerDependencies": {
"@tangle-network/agent-eval": ">=0.117.1 <0.118.0",
"@tangle-network/agent-eval": ">=0.120.1 <0.121.0",
"@tangle-network/agent-interface": ">=0.25.0 <0.26.0",
"@tangle-network/sandbox": ">=0.8.0 <1.0.0",
"playwright": "^1.40.0"
Expand Down
54 changes: 47 additions & 7 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 11 additions & 8 deletions src/improvement/agentic-generator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ import { existsSync, readFileSync, rmSync } from 'node:fs'
import { join, resolve, sep } from 'node:path'
import type {
AnalystFinding,
CostLedger,
CostLedgerHandle,
CostReceipt,
CostReceiptInput,
MaximumCharge,
Expand Down Expand Up @@ -246,10 +246,14 @@ export function agenticGenerator(opts: AgenticGeneratorOptions = {}): CandidateG
costLedger,
costPhase,
}) {
if (opts.codexReproducible && !costLedger) {
throw new Error(
'agenticGenerator: reproducible Codex requires the run-wide CostLedger supplied by agent-eval 0.117+',
)
let reproducibleCostLedger: CostLedgerHandle | undefined
if (opts.codexReproducible) {
if (!costLedger) {
throw new Error(
'agenticGenerator: reproducible Codex requires the run-wide CostLedger supplied by agent-eval 0.117+',
)
}
reproducibleCostLedger = costLedger
}
const basePrompt = appendProfileResourcePaths(
buildPrompt({ report, findings }),
Expand Down Expand Up @@ -314,13 +318,12 @@ export function agenticGenerator(opts: AgenticGeneratorOptions = {}): CandidateG
return harnessResult
}

if (opts.codexReproducible) {
const ledger = costLedger as CostLedger
if (reproducibleCostLedger) {
const model = opts.profile?.model?.default
if (!model) {
throw new Error('agenticGenerator: reproducible Codex requires profile.model.default')
}
const paid = await ledger.runPaidCall({
const paid = await reproducibleCostLedger.runPaidCall({
channel: 'driver',
phase: costPhase ?? 'search.proposal',
actor: `agentic-generator:${harness}`,
Expand Down
1 change: 0 additions & 1 deletion src/improvement/improve.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -474,7 +474,6 @@ describe('improve() — default proposer resolution (substrate export drift guar
onProvenance: () => {
provenanceCalls += 1
},
collectWorkerRecords: () => [],
expectUsage: 'assert',
captureSource: 'eval-run',
autoOnPromote: 'none',
Expand Down
4 changes: 2 additions & 2 deletions src/improvement/improvement-driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
*/

import { spawnSync } from 'node:child_process'
import type { AnalystFinding, CostLedger } from '@tangle-network/agent-eval'
import type { AnalystFinding, CostLedgerHandle } from '@tangle-network/agent-eval'
import {
type CodeSurface,
type LabeledScenarioStore,
Expand Down Expand Up @@ -66,7 +66,7 @@ export interface CandidateGenerator {
generation?: number
candidateIndex?: number
/** Shared run-wide paid-call account supplied by agent-eval 0.117+. */
costLedger?: CostLedger
costLedger?: CostLedgerHandle
/** Receipt attribution phase supplied alongside `costLedger`. */
costPhase?: string
}): Promise<{ applied: boolean; summary: string }>
Expand Down
35 changes: 35 additions & 0 deletions tests/improve.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,41 @@ describe('improve() — facade over selfImprove', () => {
expect(out.profile.prompt?.systemPrompt).toBe('BASELINE')
})

it('forwards dispatchTimeoutMs through improve() to abort a stalled cell', async () => {
let observedAbort = false
const stalledAgent = (
_surface: MutableSurface,
_scenario: DemoScenario,
ctx: DispatchContext,
): Promise<string> =>
new Promise(() => {
ctx.signal.addEventListener(
'abort',
() => {
observedAbort = true
},
{ once: true },
)
})

await expect(
improve(profileWith('BASELINE'), [], {
surface: 'prompt',
gate: 'none',
generator: scriptedWinner,
scenarios: scenarios.slice(0, 2),
judge,
agent: stalledAgent,
budget: { holdoutFraction: 0.5 },
dispatchTimeoutMs: 25,
expectUsage: 'off',
maxConcurrency: 1,
}),
).rejects.toThrow(/dispatch exceeded 25ms/)

expect(observedAbort).toBe(true)
})

it('applies the promoted prompt back into the profile on a ship verdict', async () => {
const profile = profileWith('BASELINE')
const out = await improve(profile, [], {
Expand Down
Loading