perf(sessions): cap warm session runtimes to bound /sessions memory growth#9984
Draft
Guilhem-lm wants to merge 1 commit into
Draft
perf(sessions): cap warm session runtimes to bound /sessions memory growth#9984Guilhem-lm wants to merge 1 commit into
Guilhem-lm wants to merge 1 commit into
Conversation
…rowth The /sessions sidebar eagerly warmed a full runtime (AIChatManager + loaded chat history + an IndexedDB connection) for every visible session, so the page's mounted weight grew linearly (O(number of non-archived in-family sessions)) and never released as sessions accumulate over time. Cap the warm runtimes with an MRU (the active session + up to MAX_WARM_RUNTIMES) and dispose the rest, mirroring the existing preview-tab and editor MRU caps. Sessions holding volatile state — a mid-stream generation, a pending tool confirmation, or unsent composer text — are kept warm so nothing is cancelled or lost (protect-not-persist; the composer draft lives in AIChatInput local state and isn't persisted). Cold (evicted) sessions show the neutral status dot with no unread badge until reopened. Measured ~1.5-2x the transcript size per resident session; the change converts O(N) growth into O(1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deploying windmill with
|
| Latest commit: |
086a2c3
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a72cda5a.windmill.pages.dev |
| Branch Preview URL: | https://glm-cap-warm-session-runtime.windmill.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
/sessionspage (session-v2 sidebar) eagerly warmed a full runtime — anAIChatManagerwith its whole loaded chat history, plus an IndexedDB connection — for every visible session, and rendered one mounted chat per warm runtime. Mounted weight therefore grew linearly with the number of non-archived in-family sessions and never released, so the page got progressively heavier as sessions accumulate over time.This caps the warm runtimes with an MRU (like the existing preview-tab
MAX_MOUNTED_TABSand editorMAX_WARM_EDITORScaps) and evicts the rest, converting O(number of sessions) → O(1) resident weight.Measured
With
performance.measureUserAgentSpecificMemory()(GC-forced), 10 seeded sessions of ~4.5 MB transcript each: ~7 MB marginal heap per extra resident session (~1.5–2× the transcript JSON). The win is bounding unbounded growth rather than a large absolute number for light use.Changes
sessionRuntime.svelte.ts: addMAX_WARM_RUNTIMESMRU (runtimeWarmIds/promoteRuntimeWarm/evictColdRuntimes), mirroringeditorWarmIds.disposeRuntimedrops MRU/draft bookkeeping.evictColdRuntimeskeeps a runtime warm when it holds volatile state that would otherwise be lost — a mid-stream generation (manager.loading, never cancelled), a pending tool confirmation (lasttoolmessage withneedsConfirmation), or unsent composer text. The composer draft lives inAIChatInputlocal state (notmanager.instructions), so it's tracked via the existingonDraftChangehook →setSessionDraftFlag→ asessionsWithDraftset.SessionPicker.svelte: remove the eager warm-every-session effect. Cold (evicted) sessions render the neutral status dot and no unread badge until reopened.sessions/+page.svelte: promote the active session into the MRU on arrival; an eviction effect runs on session switch and when a background stream settles.SessionWrapper.svelte/sessionUnread.svelte.ts: wire the draft flag; revert unread to live-only.No visible UI change
This is a lifecycle/memory optimization — no visible UI effect for the common case, so no screenshots. The one behavioral change: a cold (evicted, idle) session shows the neutral dot and no unread badge until it's reopened.
Test plan
document.querySelectorAll('[role=\"region\"][aria-label=\"AI chat\"]').lengthcaps) and heap stays roughly flat instead of climbing per session.npm run checkpasses.