Skip to content

perf(adapter): opt into ?include_skills=false on batch conversation fetch (-98% wire bytes)#651

Closed
simonrosenberg wants to merge 1 commit into
mainfrom
perf/skill-trim-include-skills
Closed

perf(adapter): opt into ?include_skills=false on batch conversation fetch (-98% wire bytes)#651
simonrosenberg wants to merge 1 commit into
mainfrom
perf/skill-trim-include-skills

Conversation

@simonrosenberg
Copy link
Copy Markdown
Contributor

@simonrosenberg simonrosenberg commented May 20, 2026

Draft — depends on companion PRs landing first; opens now so the typed code path is end-to-end runnable, lintable, and profilable.

What this does

Canvas only reads agent.kind and agent.llm.model from GET /api/conversations responses (see toAppConversation). The ~260 KB agent.agent_context.skills payload that load_user_skills=true / load_public_skills=true agents accumulate is parsed + structurally-shared on every conversation poll but never accessed client-side.

Pass { includeSkills: false } to ConversationClient.getConversations so the agent-server trims the payload at the route boundary. The local agent-server runtime keeps the full skill catalog server-side for prompt rendering — pure client-side perf with no functional change.

Companion PRs (currently pinned via SHA)

  • software-agent-sdk#3316 — server-side ?include_skills=false opt-in. The agent-server side is the load-bearing change; this needs to ship first.
  • typescript-client#172 (draft) — typed includeSkills option on ConversationClient. Pinned via SHA 6aa5593 in this PR's package.json.

When both companions release, the next step is:

  1. Flip the typescript-client SHA pin to a normal version tag.
  2. Mark this PR ready for review + merge.

Until then the SHA pin gives us a working end-to-end build: CI compiles the typed code path against the actual library, tests run, profile probes can hit the live agent-server with both companions applied locally.

Measured impact (honest)

Re-ran the empirical profile against the live agent-server with this PR applied. Wire-level reduction is rock-solid; user-perceived cold-open does NOT measurably improve on a fast localhost dev box. The earlier "-42% cold-open" claim was a Vite cold-start artifact in the baseline measurement, not the trim's effect.

Wire (reproducible):

Endpoint Default ?include_skills=false Delta
GET /api/conversations (40-skill conv) 265 KB, 3.3 ms server 4.7 KB, 2.4 ms server -98% bytes, -27% server time

Browser cold-open wall-clock (3 runs each, Vite warmup excluded):

Conversation Baseline (no opt-in) Opt-in Delta
Heavy (40 skills, 161 events) 1131, 1132, 1200 ms 1155, 1184 ms within run-to-run noise
Lean (0 skills, 0 events) 1030 ms 1002 ms within noise

The cold-open is dominated by a ~1100 ms app-boot floor (React mount + chat shell + event render). The 260 KB JSON parse + React Query structural-share that I thought was the bottleneck takes ~5-10 ms on V8 — invisible against the app-boot floor.

Where the trim actually wins:

  • Network bandwidth on cloud backends (100ms+ RTT) — full 260 KB payload per fetch is a real cost; 4.7 KB is not. Big difference on slow links or metered bandwidth.
  • Sustained polling loaduseUserConversation refetches via refetchInterval. Over a long session the cumulative bytes drop ~98%. Matters more for memory / battery than for first paint.
  • Server-side CPU — 27% less serialization time per request. Adds up across many concurrent clients.
  • React Query memory — 260 KB × N conversations held in cache → ~5 KB × N. Meaningful for users with many open conversations.

Where it doesn't win:

  • Cold-open wall-clock on a fast localhost dev box. End-users on localhost won't feel a difference. The dev experience is the same.

Test plan

  • npx vitest run __tests__/api/agent-server-adapter.test.ts — 42 passed.
  • npx eslint + npx tsc --noEmit — clean.
  • End-to-end against canvas with both companions running locally — wire payload drops 265 KB → 4.7 KB; default response (no param) unchanged.

🤖 Generated with Claude Code

…etch

Canvas only reads ``agent.kind`` and ``agent.llm.model`` from
``GET /api/conversations`` responses (see ``toAppConversation``) — the
~260 KB ``agent.agent_context.skills`` payload that
``load_user_skills=true`` / ``load_public_skills=true`` agents
accumulate is never accessed client-side, but every conversation
poll has to parse + structurally-share it, which is the bottleneck
the perf profiling identified.

Pass ``{ includeSkills: false }`` to ``ConversationClient.getConversations``
so the agent-server trims the payload at the route boundary. The
local agent-server runtime keeps the full skill catalog server-side
for prompt rendering — pure client-side perf with no functional
change.

Companion PRs (both pinned via SHA temporarily so this PR is
end-to-end runnable / lintable / profileable; SHAs flip to released
versions once both companions ship):

- software-agent-sdk#3316: server-side ``include_skills`` query param
- typescript-client#172: typed ``includeSkills`` option on
  ``ConversationClient``, pinned via the SHA in package.json

Measured impact (full empirical profile in
software-agent-sdk#3301): ~260 KB → ~5 KB on the wire,
``first event painted`` 2226 ms → 1283 ms (-42%) for a 40-skill
conversation cold-open.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-canvas Ready Ready Preview, Comment May 20, 2026 9:11am

Request Review

@simonrosenberg
Copy link
Copy Markdown
Contributor Author

Closing as obsolete. The companion SDK PR OpenHands/software-agent-sdk#3316 was reframed as a breaking change — the agent-server now trims agent.agent_context.skills from conversation responses by default. Canvas doesn't read this field, so once the new SDK ships there's no canvas code change needed.

The legacy opt-in (?include_skills=true) is documented in the SDK PR for the rare caller that needs the prior shape.

Cumulative work on this branch isn't lost — when the SDK release lands, all we need to do is bump the bundled image / dependency version. No code change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant