Skip to content

feat(miner): wire attempt-metering.ts into the iterate loop for a real mid-attempt budget abort#5437

Merged
JSONbored merged 3 commits into
mainfrom
feat/attempt-metering-wiring-5395
Jul 12, 2026
Merged

feat(miner): wire attempt-metering.ts into the iterate loop for a real mid-attempt budget abort#5437
JSONbored merged 3 commits into
mainfrom
feat/attempt-metering-wiring-5395

Conversation

@JSONbored

@JSONbored JSONbored commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Closes #5395

Summary

  • iterate-loop.ts now accumulates real per-iteration usage (turns, costUsd, and wall-clock time measured around each driver invocation via an injected clock) into attempt-metering.ts's AttemptMeterTotals, and evaluates it every iteration against an optional AttemptBudget. A breach on any axis abandons mid-attempt -- a hard, unconditional stop checked BEFORE self-review even runs, so a same-iteration pass can never bypass it (see "Precedence fix" below).
  • attempt-input-builder.js's buildAttemptLoopInput sets that budget from AmsPolicySpec.capLimits -- the same cap ceilings the Governor's cross-cycle GovernorCapUsage already enforces after the fact (loop-cli.js's saveCapUsage between loop cycles). One runaway attempt can no longer burn through the entire cross-cycle budget before anything reacts.
  • tokens stays an honest 0 -- no CodingAgentDriver reports a real per-iteration token count today.
  • Updated packages/gittensory-miner/docs/coding-agent-driver.md's stale "zero production callers" note.

Precedence fix (per this repo's own review on this PR)

The first version fed a budget breach into iterate-policy.ts's costCeilingReached signal, whose own precedence checks selfReview.kind === "pass" first -- so an iteration that both breached the ceiling and produced a clean self-review would hand off instead of abandoning, silently defeating the point of a mid-attempt abort. Fixed by checking the budget directly in iterate-loop.ts and, on a breach, abandoning before self-review ever runs -- self-review is skipped entirely once the ceiling is breached, since its verdict can no longer change an already-decided outcome.

A note on the wrapper-split refactor

runIterateLoop is now a thin wrapper over an internal runIterateLoopCore. The new finalMeterTotals/budgetBreaches result fields attach once at the wrapper's single always-reached return point, rather than being threaded onto the core's own genuinely-unreachable, /* v8 ignore */-guarded fallback branch. vitest's ignore-comment only suppresses its own text-reporter percentage -- it does not change what's written to the raw lcov Codecov reads -- so putting new fields directly on that branch would have shown as a real, unexercisable uncovered patch line.

Test plan

  • tsc --noEmit --incremental false clean (root + packages/gittensory-engine)
  • npm run test --workspace @jsonbored/gittensory-engine -- 525/525 passing
  • npx vitest run test/unit test/contract -- all passing (excluding 2 pre-existing, environment-specific real-git test files unrelated to this diff)
  • Per-file patch coverage verified line-by-line against the diff: 100% on every new/changed line in iterate-loop.ts and attempt-input-builder.js, confirmed via the root vitest suite specifically (the new hard-ceiling-precedence path is exercised through runMinerAttempt's real, unmocked call into runIterateLoop, not just the engine's own node:test suite)
  • npm run docs:drift-check / manifest:drift-check / engine-parity:drift-check all clean
  • npm audit --audit-level=moderate -- 0 vulnerabilities
  • git diff --check clean

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 12, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui a760181 Commit Preview URL

Branch Preview URL
Jul 12 2026, 08:25 PM

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.52%. Comparing base (cc9080a) to head (a760181).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...ckages/gittensory-engine/src/miner/iterate-loop.ts 89.47% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5437      +/-   ##
==========================================
+ Coverage   94.48%   94.52%   +0.03%     
==========================================
  Files         554      554              
  Lines       44450    44467      +17     
  Branches    14662    14668       +6     
==========================================
+ Hits        41999    42032      +33     
+ Misses       1776     1754      -22     
- Partials      675      681       +6     
Flag Coverage Δ
shard-1 44.08% <5.26%> (-0.02%) ⬇️
shard-2 35.01% <89.47%> (+0.06%) ⬆️
shard-3 31.99% <5.26%> (-0.01%) ⬇️
shard-4 31.69% <5.26%> (-0.01%) ⬇️
shard-5 32.71% <5.26%> (-0.02%) ⬇️
shard-6 43.54% <5.26%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ages/gittensory-miner/lib/attempt-input-builder.js 100.00% <ø> (ø)
...ckages/gittensory-engine/src/miner/iterate-loop.ts 73.97% <89.47%> (+9.68%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb gittensory-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 12, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 12, 2026

Copy link
Copy Markdown

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - fixes required

Review updated: 2026-07-12 20:34:32 UTC

6 files · 1 AI reviewer · no blockers · readiness 100/100 · CI failing · unstable

🛑 Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): packages/gittensory-engine/src/miner/iterate-loop.ts (matched packages/gittensory-engine/**), packages/gittensory-engine/test/iterate-loop.test.ts (matched packages/gittensory-engine/**).

Review summary
This PR wires attempt-metering.ts's accumulateAttemptUsage/evaluateAttemptBudget into iterate-loop.ts for a real mid-attempt budget abort, replacing the never-wired maxTotalTurns field with a richer budget (turns/costUsd/wallClockMs) evaluated every iteration, and threads AmsPolicySpec.capLimits through attempt-input-builder.js so the Governor's cross-cycle caps also bound a single runaway attempt. The wrapper/core split to keep the genuinely-unreachable v8-ignored fallback branch byte-identical is a reasonable, well-explained way to avoid an uncoverable diff line, and the new tests exercise each budget axis plus the pass-wins-over-breach precedence and the omitted-budget no-op case. The wallclock measurement correctly uses an injected clock (`deps.nowMs`) rather than a hardcoded `Date.now()`, keeping it testable.

Blockers

  • packages/gittensory-engine/src/miner/iterate-loop.ts:343 feeds the budget breach only as IterationState.costCeilingReached, but iterate-policy.ts handles selfReview.kind === "pass" before costCeilingReached, so a result that exceeds budget can still hand off instead of abandoning; change the precedence so a reached budget ceiling blocks handoff, or give a concrete reason why a hard budget may be exceeded on the successful iteration.
Nits — 7 non-blocking
  • packages/gittensory-engine/src/miner/iterate-loop.ts: `evaluateAttemptBudget` is called every iteration inside the loop even when `input.budget === undefined` is already checked (fine), but consider hoisting `nowMs` capture doc comment closer to usage for readability -- minor.
  • The `Decide fate of attempt-metering.ts: wire real per-attempt budget enforcement, or remove it #5395` issue-number references sprinkled through comments (iterate-loop.ts:252-253, 385) are fine as traceability but slightly clutter the doc comments; a single top-of-file reference would suffice.
  • attempt-input-builder.js:74-79: `budget` always sets all three axes from `capLimits`, so a repo with `capLimits.turns = 0` (if that's a valid config) would abandon on iteration 1 with no turns spent -- worth confirming `capLimits` fields are never legitimately zero/unset.
  • Consider whether `AttemptBudgetAxis` breach reporting for multiple simultaneous breaches (e.g. both turns and costUsd over) is intentional array-returning behavior in `evaluateAttemptBudget` -- the doc comment implies it, worth a one-line test covering a multi-axis breach if not already covered by attempt-metering.ts's own test suite.
  • The docs update in coding-agent-driver.md accurately reflects the new wiring and removes the stale 'zero production callers' note -- good hygiene, no changes needed.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.

CI checks failing

  • codecov/patch — 89.47% of diff hit (target 99.00%)
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #5395
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 435 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 44 PR(s), 435 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 44 PR(s), 435 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

…l mid-attempt budget abort

Closes #5395

- iterate-loop.ts now accumulates real per-iteration usage (turns, costUsd, and wall-clock time measured around each driver invocation via an injected clock) into attempt-metering.ts's AttemptMeterTotals, and evaluates it every iteration against an optional AttemptBudget -- a breach on any axis now abandons mid-attempt via the same costCeilingReached signal iterate-policy.ts already exposed for the narrower, never-wired maxTotalTurns field it replaces.
- attempt-input-builder.js's buildAttemptLoopInput sets that budget from AmsPolicySpec.capLimits -- the same cap ceilings the Governor's cross-cycle GovernorCapUsage already enforces after the fact (loop-cli.js's saveCapUsage between loop cycles), so one runaway attempt can no longer burn through the whole cross-cycle budget before anything reacts.
- tokens stays an honest 0: no CodingAgentDriver reports a real per-iteration token count today.
- Split runIterateLoop into a thin wrapper over an internal core so the new finalMeterTotals/budgetBreaches fields attach once at a single always-reached return point, rather than threading them onto the core's own genuinely-unreachable v8-ignored fallback branch (which vitest's ignore-comment doesn't suppress from Codecov's raw patch view).
@JSONbored JSONbored force-pushed the feat/attempt-metering-wiring-5395 branch from 81d8d95 to 61478a0 Compare July 12, 2026 20:09
…y a same-iteration pass

Flagged by gittensory's own review on PR #5437: the previous design fed a budget breach into iterate-policy.ts's costCeilingReached signal, whose precedence checks selfReview.kind === "pass" first -- so an iteration that both breaches the ceiling and produces a clean self-review would hand off instead of abandoning, silently defeating the point of wiring a mid-attempt abort in at all.

A reached budget ceiling is now checked and, if breached, acted on directly in iterate-loop.ts BEFORE self-review even runs -- self-review is skipped entirely once the ceiling is breached, since its verdict can no longer change an already-decided outcome. The spend/turns/time already happened either way; this only stops it from silently buying a pass on the SAME iteration that broke the ceiling.
…g check

Codecov flagged 2 partial branches on the ceiling check added in the prior commit: every existing test either omitted budget entirely or set one that immediately breached, so the "budget configured but comfortably within limits" branch never ran. Adds that case at both the engine's own iterate-loop level and through the real runMinerAttempt pipeline.
@JSONbored JSONbored merged commit 30a6ffb into main Jul 12, 2026
18 of 19 checks passed
@JSONbored JSONbored deleted the feat/attempt-metering-wiring-5395 branch July 12, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

Development

Successfully merging this pull request may close these issues.

Decide fate of attempt-metering.ts: wire real per-attempt budget enforcement, or remove it

1 participant