feat(desktop): acp tool summaries (experiment)

[tellaho]

Category: improvement
User Impact: Users can opt into friendlier agent activity rows that describe tool intent while the production default stays unchanged.
Problem: Tool activity rows currently expose implementation details like raw commands and file paths, which makes busy agent turns harder to scan. Because this touches production agent activity and adds an extra fast summary call, it needs a safe default-off rollout path.
Solution: Buzz now emits and consumes per-tool friendly ACP titles, but the visible summaries and background summary generation are both gated behind the acpToolSummaries preview experiment. When the experiment is off, compact rows keep the old raw labels and spawned agents receive BUZZ_AGENT_NO_TOOL_SUMMARY=1; when it is on, friendly labels paint while failures and user opt-outs still take precedence.

File changes

crates/buzz-agent/src/agent.rs
Adds deterministic friendly fallback titles and an async fast-pass summary update for individual tool calls. Keeps exact tool identity in toolName and sends summary updates without status so they cannot clobber tool state.

crates/buzz-agent/src/config.rs
Adds BUZZ_AGENT_NO_TOOL_SUMMARY and BUZZ_AGENT_TOOL_SUMMARY_MODEL configuration for disabling or redirecting the fast summary pass.

crates/buzz-agent/src/llm.rs
Carries the tool-summary config through LLM test/default setup so summary behavior can be isolated in tests.

crates/buzz-agent/tests/golden_transcripts.rs
Adds transcript coverage for pending friendly titles, title-only async summary updates, and the kill switch path.

crates/buzz-agent/tests/regressions.rs
Disables summary generation in request-count-sensitive regression harnesses so existing LLM sequencing remains deterministic.

desktop/scripts/check-file-sizes.mjs
Updates documented size-budget exceptions for files that gained focused gate/runtime logic.

desktop/src-tauri/src/commands/agent_settings.rs
Registers the desktop experiment mirror command so the frontend can persist preview toggle state for Rust spawn code.

desktop/src-tauri/src/lib.rs
Exposes the experiment mirror command to the Tauri invoke layer.

desktop/src-tauri/src/managed_agents/experiments.rs
Adds the desktop-side experiment mirror store, atomic writes, fail-safe parsing, and tests for missing/malformed state.

desktop/src-tauri/src/managed_agents/mod.rs
Exports the experiment helpers and the acpToolSummaries experiment id for managed-agent spawn code.

desktop/src-tauri/src/managed_agents/runtime.rs
Applies the spawn-time cost gate: experiment off forces BUZZ_AGENT_NO_TOOL_SUMMARY=1, experiment on leaves user opt-outs untouched.

desktop/src-tauri/src/managed_agents/runtime/tests.rs
Adds spawn-env tests covering default off, enabled behavior, and user-provided opt-out precedence.

desktop/src/app/AppShell.tsx
Mirrors preview experiment state to Rust on app boot and whenever the toggle state changes.

desktop/src/features/agents/ui/AgentSessionToolItem/CompactToolSummaryRow.render.test.mjs
Adds render-level coverage proving visible row labels change only when summaries are enabled and failures still win.

desktop/src/features/agents/ui/AgentSessionToolItem/CompactToolSummaryRow.tsx
Renders friendly summary titles ahead of action descriptors when enabled, while preserving exact receipts on hover/expand.

desktop/src/features/agents/ui/AgentSessionToolItem/ToolItem.tsx
Threads the preview flag into compact tool summary building for the main activity row surface.

desktop/src/features/agents/ui/agentSessionToolSummary.test.mjs
Adds unit coverage for summary-title off/on behavior and failure precedence at the summary-builder choke point.

desktop/src/features/agents/ui/agentSessionToolSummary.ts
Stores summaryTitle separately from raw labels and makes friendly title usage explicit through a default-off option.

desktop/src/features/agents/ui/agentSessionTranscript.test.mjs
Adds transcript ingestion tests for ACP summary updates, out-of-order updates, and fallback behavior.

desktop/src/features/agents/ui/agentSessionTranscript.ts
Ingests ACP tool_call_update summary titles into transcript state without mutating status, args, result, or timestamps.

desktop/src/features/agents/ui/agentSessionTranscriptHelpers.ts
Adds helper logic for recognizing Buzz ACP tool-summary update metadata.

desktop/src/features/agents/ui/agentSessionTranscriptPresentation.test.mjs
Adds presentation-level coverage for headline gating behavior.

desktop/src/features/agents/ui/agentSessionTranscriptPresentation.ts
Threads summary-title gating into activity headline generation so compact feed headlines match row behavior.

desktop/src/features/agents/ui/agentSessionTypes.ts
Extends transcript tool state with separate summary-title data.

desktop/src/features/channels/ui/BotActivityBar.tsx
Reads the preview flag for composer activity headlines so the live activity bar matches the tool-row experiment state.

desktop/src/shared/api/tauri.ts
Adds setDesktopExperiments for mirroring preview toggle state to Rust.

desktop/src/shared/features/index.ts
Exports the new experiment mirror hook from the shared feature module.

desktop/src/shared/features/useDesktopExperimentsMirror.ts
Adds the React hook that observes preview feature overrides and mirrors them to the Tauri command.

preview-features.json
Adds the default-off acpToolSummaries preview experiment.

Reproduction Steps

1. Build and run the desktop app from this branch.
2. Leave Friendly Tool Summaries disabled in Settings → Experiments and run an agent turn with shell/file-read tools.
3. Confirm compact activity rows keep the previous raw labels, such as Ran git status and Read crates/buzz-agent/src/agent.rs.
4. Start or restart an agent with the experiment disabled and confirm the spawned environment includes BUZZ_AGENT_NO_TOOL_SUMMARY=1.
5. Enable Friendly Tool Summaries, restart the agent, and run the same style of tool calls.
6. Confirm compact rows now show intent-level labels such as checking repository state, while expanding or hovering still exposes the exact command/path.
7. Trigger a failed shell tool and confirm the failure label still wins over any friendly phrase.

Screenshots/Demos

Screenshots from Marge's full UI pass against a dist rebuilt from gate commit 96a83a1bf. The only change between each pair is the acpToolSummaries preview flag.

1. Activity panel — tool rows (ToolItem / CompactToolSummaryRow)

OFF — raw compact labels:

ON — friendly intent summaries:

2. Composer activity headline (BotActivityBar / getActivityHeadline)

OFF — raw headline (Ran command · git status):

ON — friendly headline (checking repository state · git status):

Validation

• pnpm test 1937/1937
• desktop cargo test --lib 978 pass
• cargo test -p buzz-agent
• pnpm exec tsc --noEmit
• biome check .
• cargo fmt

Rollout note: cut production builds from 96a83a1bf or newer. A stale dist/ built before the gate does not contain acpToolSummaries and can make off/on testing misleading.

Coordination: buzz://message?channel=8905fd94-5537-4062-8d45-1d79eff51a74&thread=6508f23bd67de9b91443714c5be913e0cae949c5c98dee393a5822620edd15e1

[wesbillman]

Approve. Read the full 32-file diff against merge-base and traced both the Rust and TS paths. CI note first: the two red checks (Desktop Smoke E2E (1) and its Desktop roll-up) were an apt-mirror infra flake (packages.microsoft.com ... Clearsigned file isn't valid while installing Playwright browser deps) — I reran the failed jobs and everything is green now, including the new golden transcripts and spawn-env tests.

What I verified:

• Two-stage title design is sound. Stage 1 (emit_pending) is deterministic and immediate; stage 2 (spawn_tool_summary) publishes a title-only tool_call_update tagged _meta.buzz.toolSummary with deliberately no status field, and the golden transcript test asserts that (update.get("status").is_none()). On the ingest side, extractBuzzToolSummaryTitle routes tagged updates to applyToolSummaryTitle, which only sets summaryTitle — it never touches upsertTool's status merge, so a late summary can't resurrect or clobber a terminal row. Out-of-order summaries are dropped, which is the right call for a cosmetic title.
• The gate is genuinely fail-safe. resolve_experiment_enabled is enabled only on explicit true; missing file, malformed JSON, absent key, and non-boolean values all resolve off (tested). Spawn gate writes BUZZ_AGENT_NO_TOOL_SUMMARY=1 after user env so the off-state is authoritative, and the on-state leaves a user opt-out intact (tested both ways in runtime/tests.rs).
• Failure precedence holds at both layers. The builder nulls summaryTitle on failure, and CompactToolSummaryRow independently bypasses both the summary span and the action descriptor when failed — so even a future builder regression can't paint a friendly phrase over an error. Render test covers it.
• The restart-confirm ordering is correct and well-reasoned: toggle → awaited explicit mirror write → snapshot respawn, precisely because the passive useDesktopExperimentsMirror effect is unordered relative to the respawn. Partial-failure messaging doesn't roll back the toggle, matching the stated best-effort convention.

Non-blocking observations:

1. Stage 1 is ungated by design — worth keeping in mind at rollout. Even with the experiment off, the pending tool_call now carries the friendly deterministic title (with exact identity preserved in toolName). The desktop UI is unaffected (labels come from the classifier/status path, confirmed by the OFF screenshots), but any other ACP client that renders title verbatim will see the new phrasing regardless of the experiment. The PR body says "emits and consumes" but the emission half isn't behind the flag — intentional as far as I can tell, just not obvious from the rollout note.
2. Summary cost defaults to the session model. With the experiment on, each tool call spends up to 2 extra LLM calls (retry) on the session's effective model unless BUZZ_AGENT_TOOL_SUMMARY_MODEL is set. Before this graduates beyond preview, consider defaulting the summary pass to a designated cheap/fast model rather than opt-in via env.
3. check-file-sizes.mjs overrides grew again (runtime.rs 2213→2236, tauri.ts 1380→1391, agentSessionTranscript.ts 1167→1192). The entries are documented with queued splits per repo convention, but the queue is getting long — the transcript split in particular looks overdue.

Nice test discipline across the stack: wire-level golden transcripts, pure-logic node tests for the restart flow, and render-level coverage at the row. Ship it.

[wpfleger96]

Reviewed the full diff (not just hunks) at de96c07 across three lenses — security/reliability, design/performance, functionality/testing. The gate design is solid and fail-safe on every axis I checked, and the wire contract (_meta.buzz.toolSummary) is genuinely harness-agnostic on the consumption side, which is the right seam. One blocking issue on the generation side plus two things to clean up before this graduates from the preview experiment. Details inline.

The blocker is localized to spawn_tool_summary so it should be a quick turnaround, not a redesign.

[wpfleger96]

🔴 MUST FIX — unbounded detached summary fan-out

This fires a detached tokio::spawn for every runnable tool call before execute_parallel, so it bypasses the Semaphore::new(max_parallel_tools) bound at line 391 that gates actual tool execution. Each summary task runs a for attempt in 0..2 retry (line 690) → up to 2 provider requests each. With max_tool_calls at 64 (config.rs:557), a single busy turn can launch up to ~128 detached summary LLM requests, unbounded against max_parallel_tools, competing with the main turn for provider rate/cost budget.

Cancellation is checked between attempts and before emit, but not during the in-flight .await on llm.summarize — so once a request is issued it keeps running post-cancel until the shared ~240s LLM timeout (config.rs:719 / llm.rs:54).

Suggest routing summary jobs through a bounded semaphore/worker pool (≤ max_parallel_tools, or a lower summary-specific cap), and wrapping the .await in a select! on the cancel receiver plus a short summary-specific timeout so in-flight requests abort. Keep the retry inside that bound.

[wpfleger96]

🟡 SHOULD FIX (before GA) — tool argument values sent to the summary LLM

The fast-pass prompt embeds call.arguments (truncated to 300 chars). That raw JSON can contain a file path like ~/.ssh/id_rsa, a shell command, or a URL with an embedded key — so a fragment of a secret can reach the summary model.

This is 🟡 not 🔴 because the summary model is the same provider already receiving the full tool call in the main conversation, and the feature is default-off behind the preview experiment. But before GA I'd send only the tool name + top-level argument keys (not values) to the summary model — that's enough for a descriptive phrase ("reading config file", "querying database") with zero value leakage — or redact a key blocklist (password, key, secret, token, auth, credential).

[wpfleger96]

🟡 SHOULD FIX (before GA) — Rust experiment mirror drops the frontend override version

The frontend keys its override storage by manifest version — OVERRIDES_KEY = buzz-feature-overrides-v${manifest.version} at store.ts:12 — so a manifest bump orphans old overrides and resets the experiment to off. But this Rust mirror writes a fixed, unversioned experiments.json. On a future manifest bump, the app-startup restore path can read a stale true from the mirror and spawn agents with summaries ON while the UI now shows OFF, until the webview remirrors.

It self-heals once the webview loads and the failure mode is "on when expected off" (a cost surprise, not unsafe), hence 🟡. Suggest versioning the mirror payload or filename (e.g. experiments-v${manifest.version}.json) and ignoring mismatches, so it tracks the frontend's version semantics.

[wpfleger96]

🟢 CONSIDER (nit) — document the two-layer failure-suppression contract

Minor: the transcript stores summaryTitle unconditionally, even on failed rows — suppression of the friendly title on failure happens downstream in the render/headline layer via the !failed guard. Both current consumers (CompactToolSummaryRow, getActivityHeadline) honor it and are tested, so this is correct today. A one-line comment on the summaryTitle field pointing at the render-layer !failed guard would keep a future consumer from reading it directly and showing a friendly phrase on a failed row.

[wpfleger96]

Reviewed the full diff (not just hunks) at de96c07 across three lenses — security/reliability, design/performance, functionality/testing. The gate design is solid and fail-safe on every axis I checked, and the wire contract (_meta.buzz.toolSummary) is genuinely harness-agnostic on the consumption side, which is the right seam. One blocking issue on the generation side plus two things to clean up before this graduates from the preview experiment. Details inline.

The blocker is localized to spawn_tool_summary so it should be a quick turnaround, not a redesign.

Duplicate of review #4654643765 (accidental double-post) — dismissing; see the identical review directly above.