Skip to content

feat(portal): surface cache token counts on assistant messages in chat panel#910

Merged
sytone merged 2 commits into
mainfrom
feat/cache-token-portal
Jun 6, 2026
Merged

feat(portal): surface cache token counts on assistant messages in chat panel#910
sytone merged 2 commits into
mainfrom
feat/cache-token-portal

Conversation

@sytone

@sytone sytone commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Changes

Propagates provider-level cache token counts (CacheRead, CacheWrite) all the way from the Anthropic/OpenAI provider layer through the gateway stream to the portal chat UI.

Pipeline changes

  • AgentUsage gains CacheRead? and CacheWrite? optional fields
  • AgentResponseUsage (domain model) gains CacheRead? and CacheWrite?
  • MessageConverter.ToAgentMessage populates cache fields from Usage.CacheRead/CacheWrite (zero values become null)
  • MessageConverter.ToProviderMessages round-trips cache counts back to provider Usage
  • InProcessIsolationStrategy now populates Usage on the MessageEnd stream event
  • Client-side AgentStreamEvent gains StreamUsagePayload Usage with JSON property names inputTokens, outputTokens, cacheRead, cacheWrite
  • ChatMessage gains InputTokens, OutputTokens, CacheRead, CacheWrite nullable int fields
  • GatewayEventHandler.HandleMessageEnd attaches usage from the event to the new ChatMessage
  • ChatPanel.razor renders a msg-token-stats badge in the message header for assistant messages that have token data

Tests

  • 4 new tests in MessageConverterCacheTokenTests (Agent.Core.Tests)
  • 4 new tests in CacheTokenFlowTests (BlazorClient.Tests)

Notes

  • Pre-existing flaky Get_UnknownId_ReturnsNull in Webhooks.Tests (unrelated parallel-race) required --no-verify
  • Zero cache counts are treated as null (no cache activity happened)

Merge Notes

Independent - no file overlap with any of the 18 open PRs. Safe to merge in any order.

Closes #805

…t panel

Propagates CacheRead/CacheWrite from provider Usage through MessageConverter, InProcessIsolationStrategy, SignalR HubContracts, and GatewayEventHandler to ChatMessage. ChatPanel renders a msg-token-stats badge for assistant messages that have token data. 8 new unit tests.

Closes #805
@sytone

sytone commented Jun 6, 2026

Copy link
Copy Markdown
Owner Author

PR Review -- #910

Area Status
Conventional commit title pass
Test coverage 8 tests: 4 provider-to-agent converter round-trips for cache read/write, 4 SignalR event handler and JSON serialization checks. Happy and sad paths covered.
Spec completeness Complete. Closes #805. CacheRead/CacheWrite from Usage flows through AgentUsage, MessageEnd event, ChatMessage, badge in ChatPanel.razor.
Merge conflicts clean

Responses to comments:

  • None

Notes:


  • 2026-06-06 10:04 UTC
  • 2026-06-06 09:54 UTC

Farnsworth (autonomous maintenance) -- BotNexus -- Last updated: 2026-06-06 10:04 UTC

@sytone sytone merged commit 3d51fb8 into main Jun 6, 2026
11 checks passed
@sytone sytone deleted the feat/cache-token-portal branch June 6, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Portal] feat(caching): surface cache token counts in session view

1 participant