honesty(agent-builder): tag 15 of 20 tool options as DEMO#218
Merged
operatoruplift merged 1 commit intomasterfrom Apr 28, 2026
Merged
honesty(agent-builder): tag 15 of 20 tool options as DEMO#218operatoruplift merged 1 commit intomasterfrom
operatoruplift merged 1 commit intomasterfrom
Conversation
The /agents/builder wizard step 3 (Tools) presented 20 tools the user could attach to a custom agent: Web Search, Web Scraper, Code Executor, File System, Memory Bank, Database, HTTP/API, GitHub, Shell, Code Search, Email, Calendar, Slack, Notion, Solana, Oro GRAIL, DD.xyz Risk, x402, Image Gen, Voice. The grid was uniform — every option looked equally available and selectable. Reality: only 5 of these have a backing /api/tools/* route the agent runtime can actually call (web-search, memory, email/gmail, calendar, x402). Selecting any of the other 15 saved an entry to the agent's tool list that the runtime would never invoke, because no router exists. A user could ship an agent claiming "Slack + Notion + Database" that secretly couldn't do any of those. Three changes: 1. TOOLS list gains a `live: boolean` field. 5 marked live (web-search, memory, email, calendar, x402), 14 marked demo. Drop the 'oro-grail' row entirely (the API was retired in #209). Add a 'tokens' row (live, the existing /api/tools/tokens route). 2. Each tool tile renders a small DEMO badge when `live=false` and gets `opacity-70` so the visual hierarchy matches the truth. 3. Comment block above TOOLS spells out the contract: stay aligned with /integrations and the toolCalls router; never mark `live: true` without a working /api/tools/* route. Pure UI disclosure — no behavior change. Selecting demo tools still works (writes to the agent's localStorage entry); the user just sees the badge before they pick.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
3 tasks
operatoruplift
added a commit
that referenced
this pull request
Apr 28, 2026
…closures (#220) Two new specs in dashboard-honesty.spec.ts that prevent the recent honesty fixes from silently regressing: 1. /agents/builder Tools step - Walks the wizard: pick General Assistant -> name + description -> Tools step - Asserts the four most-recognizable stub tools (Slack, Notion, GitHub, Database) are still present - Counts DEMO badges on the page; requires >= 5 (we marked 14 stubs in #218; 5 is a defensive lower bound that protects the contract without being brittle to category copy edits) 2. /settings API Keys section - Navigates to the API tab - Asserts the disclosure copy or DEMO badge is visible (regex matches several phrasings so a future copy edit doesn't break it) - Asserts the retired "expires in 30 days" wording is absent Hermetic spec count stays at 17 — these are tests added to the existing dashboard-honesty file that's already in the CI list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
/agents/builderwizard step 3 (Tools) presented 20 tools the user could attach to a custom agent. The grid was uniform — every option looked equally available and selectable.Reality: only 5 of these have a backing
/api/tools/*route the agent runtime can actually call: web-search, memory, email/gmail, calendar, x402.Selecting any of the other 14 saved an entry to the agent's tool list that the runtime would never invoke. A user could ship an agent claiming "Slack + Notion + Database" that secretly couldn't do any of those.
Three changes:
TOOLSlist gains alive: booleanfield. 5 marked live (web-search, memory, email, calendar, x402), 14 marked demo. Drop theoro-grailrow entirely (the API was retired in honesty: retire /api/risk + /api/gold (returned fabricated demo data) #209). Add atokensrow (live, the existing/api/tools/tokensroute).live=falseand getsopacity-70so the visual hierarchy matches the truth.TOOLSspells out the contract: stay aligned with/integrationsand thetoolCallsrouter; never marklive: truewithout a working/api/tools/*route.Pure UI disclosure — no behavior change. Selecting demo tools still works (writes to the agent's localStorage entry); the user just sees the badge before they pick.
Test plan
pnpm exec tsc --noEmitcleanpnpm buildpassespnpm checkpasses