perf(tui): throttle budget/anomaly rollups off per-event path (0.1.2)#32
Merged
Conversation
…1.2) The TUI recomputed budgetStatus (up to 50k store rows) and anomalies (5k) on every event via a useMemo keyed on state.events. On a ~1GB DB a single budget query is ~1.08s; running it per event during backfill pegged the shared event loop — freezing the TUI (q unresponsive) and starving the in-process web server opened with 'w'. Recompute the store-backed rollups on a 2.5s interval tick instead of per event; the live timeline still updates instantly. No-store path unchanged.
The per-file backfill re-read the last 512KB of EVERY session file on startup (826 Claude files + 1.2GB OpenClaw on a heavy install), blocking the event loop ~8-10s — which froze the TUI and starved the in-process web server (press 'w'). Stale files (mtime older than 48h) now tail from EOF instead; their history is already in the SQLite store the timeline seeds from. Recently modified files still backfill the gap since the last run. Shared helper backfillStartOffset() in util/backfill.ts, used by the claude-code, openclaw, and codex adapters. Smoke (1GB history): in-process web during startup 19s -> ~1s -> 0.34s.
Addresses Codex [P2]: skipping stale (>48h) files' backfill caused data loss on first run / empty / unavailable / freshly-pruned store — those files were never ingested, so their events were dropped (the store seed only helps after a prior successful ingestion). Now the stale-skip is enabled only when the store seed returns >0 events (setStaleSkipEnabled in util/backfill.ts, called from both the serve and TUI startup paths). Fresh/empty store -> full backfill (one-time, complete); returning users -> fast skip. Default off keeps adapter unit tests intact.
Two remaining loop-blockers eliminated so the TUI + in-process web stay responsive throughout startup, not just after it settles: 1. Adapter initial-scan backfill now drains one file per macrotask (util/backfill-queue.ts) instead of a synchronous file-read storm, so HTTP/SSE interleave during startup. Live post-scan events still inline. 2. Budget rollup aggregates in SQL (store.budgetRollup: SUM today's cost + top session by total) — ~20ms vs ~1s pulling 50k rows into JS every 2.5s tick. Breach logic shared via budgets.budgetStatusFromTotals so in-memory and SQL paths can't diverge. Smoke (1GB history): sub-100ms steady-state; web usable in ~3s (was minutes).
mishanefedov
added a commit
that referenced
this pull request
May 25, 2026
perf(tui): throttle budget/anomaly rollups off per-event path (0.1.2)
mishanefedov
added a commit
that referenced
this pull request
May 25, 2026
perf(tui): throttle budget/anomaly rollups off per-event path (0.1.2)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
budgetStatus(up to 50k SQLite rows) andanomalies(5k) on every event (useMemo keyed onstate.events).qunresponsive) and starving the in-process web server opened withw(minutes to load).Test plan
npm run typecheckcleannpm test— 404 passednpm run buildOK