refactor(agent): simplify tool disclosure, index tool schemas#94
Merged
Conversation
Progressive tool disclosure exposed three meta-tools; tool_describe was a separate round-trip to fetch a tool's parameter schema after tool_search. Collapse to two: tool_search now returns each hit's parameter schema alongside name and description, so the model can go straight to tool_call. Also fold parameter names, descriptions and enum values into the BM25 index body so discriminating keywords that live in the schema (a repo owner, a channel id, an image size) lift recall. Lower the default search_result_limit from 10 to 5 to offset the larger per-hit payload. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
The prior commit lowered tool_search's default result count to 5 to offset the larger per-hit payload from folding parameter schemas into hits. Restore it to 10: recall from more candidates outweighs the marginal payload, and a caller can still cap results per query via the limit argument. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
The OpenRouter-backed disclosure e2e needs a live LLM and network, so it cannot run as a deterministic unit test. The search -> call flow (hits carry parameter schemas, tool_describe removed) stays covered by the deterministic cases in tests/test_tool_search.py. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
f4e56fe to
c8ace52
Compare
Align the JSON schema maximum for the tool_search limit param with the default search_result_limit of 10, so the model cannot request more results than the configured default. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
0xKT
approved these changes
Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Progressive tool disclosure previously exposed three meta-tools:
tool_search,tool_describe, andtool_call.tool_describewas a separateround-trip to fetch a hidden tool's parameter schema after
tool_search. ThisPR collapses the flow to two meta-tools:
tool_describe.tool_searchnow returns each hit's parameterschema alongside its name and description, so the model can go straight to
tool_call. When arguments don't fit the schema the registry already returnsa correctable validation error, so the extra lookup step added no correctness.
enum values are folded into the BM25 index body (
namex3 + description +schema keywords). Discriminating keywords often live in the schema rather than
the one-line description (a repo owner, a channel id, an image size), so
indexing them lifts retrieval recall without changing the schema the model
sees. The index signature now keys on
(name, description, parameters), so aparameter change under hot-reload correctly triggers a rebuild.
Offline validation on the ToolRet benchmark (7,961 queries, 44k-tool corpus,
Raven's own BM25 kernel) shows folding the schema into the index consistently
lifts nDCG@10 by ~4-5 points over indexing name + description alone.
Type
Verification
New unit coverage: schema-keyword extraction (names, descriptions, enums,
nesting, depth cap, non-dict input), rebuild-on-parameters-change, search hits
carrying parameters, meta-tool set no longer including
tool_describe.Risk
Progressive tool disclosure is opt-in (
tools.tool_search.enableddefaults tofalse), so the default agent path is unaffected.
tool_describeis an internalmeta-tool with no public API; removing it changes only the enabled-feature flow.
tool_searchhits are larger now (they carry parameter schemas), but only whenthe model actively searches - the per-turn tool list and prompt-cache prefix are
untouched. Rollback: revert the branch.
Related Issues
N/A