Skip to content

feat(agent-interface): bind exact candidate experiments#59

Merged
drewstone merged 1 commit into
mainfrom
feat/candidate-benchmark-task
Jul 16, 2026
Merged

feat(agent-interface): bind exact candidate experiments#59
drewstone merged 1 commit into
mainfrom
feat/candidate-benchmark-task

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Problem

Evaluation, approval, and execution could describe the same candidate with different task, grader, model, workspace, and receipt identities. That made an apparent improvement impossible to prove as the exact approved run.

Solution

  • define one immutable benchmark task and experiment shared by evaluation and Runtime
  • bind execution plans to instruction delivery, resource limits, model grants, workspace identity, candidate bundles, and graders
  • require unique scenario, execution, run, receipt, and evidence identities
  • derive cost and latency from exact receipts and reject contradictory results
  • keep the greenfield candidate contract unversioned

Verification

  • pnpm check-types
  • pnpm test: 551/551 passed across 8 packages
  • pnpm build
  • pnpm --filter @tangle-network/agent-interface test: 120/120 passed after rebasing onto main
  • git merge-tree --write-tree origin/main HEAD
  • git diff origin/main...HEAD --check

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — e7bf8632

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-16T05:39:21Z

@drewstone
drewstone merged commit e1c362e into main Jul 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants