perf(adapter): opt into ?include_skills=false on batch conversation fetch (-98% wire bytes)#651
Closed
simonrosenberg wants to merge 1 commit into
Closed
perf(adapter): opt into ?include_skills=false on batch conversation fetch (-98% wire bytes)#651simonrosenberg wants to merge 1 commit into
simonrosenberg wants to merge 1 commit into
Conversation
…etch
Canvas only reads ``agent.kind`` and ``agent.llm.model`` from
``GET /api/conversations`` responses (see ``toAppConversation``) — the
~260 KB ``agent.agent_context.skills`` payload that
``load_user_skills=true`` / ``load_public_skills=true`` agents
accumulate is never accessed client-side, but every conversation
poll has to parse + structurally-share it, which is the bottleneck
the perf profiling identified.
Pass ``{ includeSkills: false }`` to ``ConversationClient.getConversations``
so the agent-server trims the payload at the route boundary. The
local agent-server runtime keeps the full skill catalog server-side
for prompt rendering — pure client-side perf with no functional
change.
Companion PRs (both pinned via SHA temporarily so this PR is
end-to-end runnable / lintable / profileable; SHAs flip to released
versions once both companions ship):
- software-agent-sdk#3316: server-side ``include_skills`` query param
- typescript-client#172: typed ``includeSkills`` option on
``ConversationClient``, pinned via the SHA in package.json
Measured impact (full empirical profile in
software-agent-sdk#3301): ~260 KB → ~5 KB on the wire,
``first event painted`` 2226 ms → 1283 ms (-42%) for a 40-skill
conversation cold-open.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
6 tasks
Contributor
Author
|
Closing as obsolete. The companion SDK PR OpenHands/software-agent-sdk#3316 was reframed as a breaking change — the agent-server now trims The legacy opt-in ( Cumulative work on this branch isn't lost — when the SDK release lands, all we need to do is bump the bundled image / dependency version. No code change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — depends on companion PRs landing first; opens now so the typed code path is end-to-end runnable, lintable, and profilable.
What this does
Canvas only reads
agent.kindandagent.llm.modelfromGET /api/conversationsresponses (seetoAppConversation). The ~260 KBagent.agent_context.skillspayload thatload_user_skills=true/load_public_skills=trueagents accumulate is parsed + structurally-shared on every conversation poll but never accessed client-side.Pass
{ includeSkills: false }toConversationClient.getConversationsso the agent-server trims the payload at the route boundary. The local agent-server runtime keeps the full skill catalog server-side for prompt rendering — pure client-side perf with no functional change.Companion PRs (currently pinned via SHA)
?include_skills=falseopt-in. The agent-server side is the load-bearing change; this needs to ship first.includeSkillsoption onConversationClient. Pinned via SHA6aa5593in this PR'spackage.json.When both companions release, the next step is:
Until then the SHA pin gives us a working end-to-end build: CI compiles the typed code path against the actual library, tests run, profile probes can hit the live agent-server with both companions applied locally.
Measured impact (honest)
Re-ran the empirical profile against the live agent-server with this PR applied. Wire-level reduction is rock-solid; user-perceived cold-open does NOT measurably improve on a fast localhost dev box. The earlier "-42% cold-open" claim was a Vite cold-start artifact in the baseline measurement, not the trim's effect.
Wire (reproducible):
?include_skills=falseGET /api/conversations(40-skill conv)Browser cold-open wall-clock (3 runs each, Vite warmup excluded):
The cold-open is dominated by a ~1100 ms app-boot floor (React mount + chat shell + event render). The 260 KB JSON parse + React Query structural-share that I thought was the bottleneck takes ~5-10 ms on V8 — invisible against the app-boot floor.
Where the trim actually wins:
useUserConversationrefetches viarefetchInterval. Over a long session the cumulative bytes drop ~98%. Matters more for memory / battery than for first paint.Where it doesn't win:
Test plan
npx vitest run __tests__/api/agent-server-adapter.test.ts— 42 passed.npx eslint+npx tsc --noEmit— clean.🤖 Generated with Claude Code