Skip to content

feat(billing): rework usage tracking + context breakdown#2373

Merged
k11kirky merged 3 commits into
mainfrom
posthog-code/usage-overhaul
May 26, 2026
Merged

feat(billing): rework usage tracking + context breakdown#2373
k11kirky merged 3 commits into
mainfrom
posthog-code/usage-overhaul

Conversation

@k11kirky
Copy link
Copy Markdown
Contributor

@k11kirky k11kirky commented May 26, 2026

Problem

Users have no visibility into how much of their LLM usage quota they've consumed until they hit a hard limit, and the existing polling-based usage fetch was inefficient and prone to drift. Free-tier users lacked a reset-time label on the sidebar bar, and the context window indicator showed only a tooltip with aggregate numbers rather than a breakdown by source.

Changes

Usage monitoring service (UsageMonitorService)

  • Introduced a new UsageMonitorService in the main process that replaces the renderer's 30-second polling loop. The service listens for LlmActivity events emitted once per completed agent turn (both Claude and Codex adapters) and coalesces bursts into a single trailing fetch per 5-second window. A 30-minute backstop timer handles idle periods and billing-period rollovers.
  • Threshold crossing detection fires ThresholdCrossed events at 50/75/90/100% for both burst and sustained buckets. Crossed thresholds are deduplicated per user/product/bucket/billing-window anchor and persisted to electron-store so notifications don't re-fire after relaunch within the same window. Stale entries are pruned on boot.
  • Added reset_at (absolute UTC timestamp) and billing_period_end to the gateway schemas so the anchor logic doesn't drift with rolling resets_in_seconds values.

tRPC surface

  • Replaced llmGateway.usage query with a dedicated usageMonitor router exposing getLatest, refresh, onUsageUpdated (subscription), and onThresholdCrossed (subscription).
  • useUsage now subscribes to onUsageUpdated and seeds the query cache from getLatest instead of polling on a timer.

Threshold toast notifications

  • initializeUsageThresholdToast subscribes to onThresholdCrossed and shows a warning toast at 50/75/90% with a reset-time label and a "View usage" action, or triggers the UsageLimitModal at 100% when a session is active.
  • Removed the old useUsageLimitDetection hook.

Sidebar usage bar

  • Shows a loading skeleton while data is in flight.
  • Displays a human-readable reset-time label (Resets in 4h 30m, Resets Jun 1 at 12:00 AM PDT, etc.) derived from the new formatResetTime utility, which prefers reset_at over resets_in_seconds.

Context breakdown popover

  • Added a ContextBreakdownPopover that replaces the plain tooltip on the ContextUsageIndicator. It shows a segmented bar and per-category token counts (System prompt, Tools, Rules, Skills, MCP, Subagents, Conversation).
  • The breakdown is computed agent-side using a character-ratio estimator (~3.5 chars/token) and emitted via _posthog/usage_update notifications. Claude estimates system prompt, CLAUDE.md rules, slash-command skills, and MCP tool metadata at session init and on changes; Codex uses a constant baseline plus the injected system prompt.
  • Extracted formatTokensCompact, getOverallUsageColor, and CONTEXT_CATEGORIES into a shared contextColors.ts utility.

Codex session-state fix

  • Replaced this.sessionState = createSessionState(...) reassignments with a new resetSessionState() mutation so the codex-client closure always writes contextUsed/contextSize to the same object reference across newSession/loadSession/resumeSession/forkSession calls.

How did you test this?

  • Added unit tests for UsageMonitorService covering threshold deduplication, cross-relaunch persistence, independent burst/sustained tracking, isPro detection, error resilience, UsageUpdated change detection, coalesce debouncing, and stop() cleanup.
  • Added unit tests for context-breakdown.ts covering all estimator functions and buildBreakdown edge cases.
  • Added unit tests for useContextUsage covering aggregate extraction, breakdown merging, and the double-underscore method prefix variant.
  • Added unit tests for ContextBreakdownPopover covering header rendering, placeholder copy, and per-category row filtering.
  • Added a regression test for createCodexClient verifying that contextUsed writes land on the correct object after resetSessionState.
  • Added unit tests for formatResetTime covering sub-hour, sub-day, multi-day, reset_at preference, and past-reset cases.

Publish to changelog?

no

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@k11kirky k11kirky marked this pull request as ready for review May 26, 2026 11:54
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 26, 2026

Prompt To Fix All With AI
Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 3
apps/code/src/renderer/features/billing/hooks/useUsage.ts:31-37
The `refetch` callback depends on the entire `refreshMutation` result object, which contains stateful fields (`isPending`, `data`, etc.) that change on every render. This causes a new `refetch` function reference on every render. Depend on just the stable `mutateAsync` function to avoid unnecessary re-renders downstream.

```suggestion
  const mutateAsync = refreshMutation.mutateAsync;
  const refetch = useCallback(async () => {
    const fresh = await mutateAsync();
    if (fresh) {
      queryClient.setQueryData(trpc.usageMonitor.getLatest.queryKey(), fresh);
    }
    return fresh;
  }, [mutateAsync, queryClient, trpc.usageMonitor.getLatest]);
```

### Issue 2 of 3
apps/code/src/renderer/features/billing/utils.test.ts:54-86
**Prefer parameterised tests**

The `formatResetTime` suite has six structurally identical tests that each supply different inputs and expected outputs. Grouping them into a single `it.each` block would make the intent clearer and make it easier to add new cases. The same pattern appears in `service.test.ts` for the threshold-crossing assertions and in `context-breakdown.test.ts` for the various estimator functions.

### Issue 3 of 3
apps/code/src/renderer/features/sessions/components/ContextBreakdownPopover.tsx:68-90
**Segment widths can exceed 100% when estimation overestimates stable tokens**

`SegmentedBar` renders each segment as `(value / total) * 100%` where `total` is `used` (the real token count). `buildBreakdown` floors `conversation` at 0, so when `stableSum > currentInputTokens` the stable categories are emitted unchanged while `conversation = 0`. The segments therefore sum to more than `total`, and each individual width exceeds its true share. The `overflow-hidden` CSS clips the bar, but the relative proportions shown to the user become misleading when estimates drift significantly above actual usage.

Reviews (1): Last reviewed commit: "feat(billing): rework usage tracking + c..." | Re-trigger Greptile

Comment thread apps/code/src/renderer/features/billing/hooks/useUsage.ts Outdated
Comment thread apps/code/src/renderer/features/billing/utils.test.ts
Comment on lines +68 to +90
);
}

function SegmentedBar({
breakdown,
total,
fallback,
}: {
breakdown: NonNullable<ContextUsage["breakdown"]>;
total: number;
fallback: string;
}) {
if (total <= 0) {
return <div className="h-1.5 w-full rounded-full bg-(--gray-4)" />;
}
return (
<div className="flex h-1.5 w-full overflow-hidden rounded-full bg-(--gray-4)">
{CONTEXT_CATEGORIES.map((cat) => {
const value = breakdown[cat.key];
if (value <= 0) return null;
return (
<div
key={cat.key}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Segment widths can exceed 100% when estimation overestimates stable tokens

SegmentedBar renders each segment as (value / total) * 100% where total is used (the real token count). buildBreakdown floors conversation at 0, so when stableSum > currentInputTokens the stable categories are emitted unchanged while conversation = 0. The segments therefore sum to more than total, and each individual width exceeds its true share. The overflow-hidden CSS clips the bar, but the relative proportions shown to the user become misleading when estimates drift significantly above actual usage.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/code/src/renderer/features/sessions/components/ContextBreakdownPopover.tsx
Line: 68-90

Comment:
**Segment widths can exceed 100% when estimation overestimates stable tokens**

`SegmentedBar` renders each segment as `(value / total) * 100%` where `total` is `used` (the real token count). `buildBreakdown` floors `conversation` at 0, so when `stableSum > currentInputTokens` the stable categories are emitted unchanged while `conversation = 0`. The segments therefore sum to more than `total`, and each individual width exceeds its true share. The `overflow-hidden` CSS clips the bar, but the relative proportions shown to the user become misleading when estimates drift significantly above actual usage.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Contributor

@jonathanlab jonathanlab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would recommend running this agents the newly merged AGENTS.md

Comment thread apps/code/src/main/services/agent/schemas.ts Outdated
Comment thread apps/code/src/main/services/llm-gateway/schemas.ts Outdated
Comment thread apps/code/src/main/services/usage-monitor/service.ts Outdated
Comment thread apps/code/src/main/trpc/routers/usage-monitor.ts Outdated
Comment thread packages/agent/src/adapters/claude/context-breakdown.ts Outdated
Comment thread apps/code/src/main/services/usage-monitor/service.ts Outdated
Comment thread apps/code/src/renderer/App.tsx Outdated
@k11kirky k11kirky added the Create Release This will trigger a new release label May 26, 2026
@k11kirky k11kirky merged commit ce80aa8 into main May 26, 2026
16 checks passed
@k11kirky k11kirky deleted the posthog-code/usage-overhaul branch May 26, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Create Release This will trigger a new release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants