Background
TokenAwareSummarizingMemory manages the context window automatically through background summarization and pruning. This works well under normal conditions. However, the system is entirely silent - users receive no signal about how full a session is, and discover the context limit only when the model returns a hard error and their message is lost.
Problem analysis
-
No emergency headroom
The memory budget is a hard 40% of the context window. When summarization is already running (summarizationInProgress = true), new messages bypass the threshold check entirely and accumulate unchecked. If the user pastes a large block of code or text during an active summarization cycle, the total can spike past the model's actual limit before the cycle finishes — resulting in a hard model error with no recovery path.
-
Memory state is completely opaque to the outside world
estimateTotalTokens() is private. There is no way for ChatViewModel, ChatSessionService, or the UI to ask "how full is this session?" The only observable signal is an exception thrown by the model. ChatState has no memory pressure field, so even if the service layer computed pressure, there would be nowhere to put it.
-
No "continue with context" workflow
When a session does hit its limit, the only option is to manually start a new session from scratch. There is no mechanism to carry the existing SessionConversationSummary - which already exists in memory and in the database - into a new session as a starting point. The structured summary is rich enough to seed a new session meaningfully, but nothing in ChatSessionService does this today.
-
Threshold constants are invisible to callers
PROTECTED_RECENT_TURNS, SUMMARIZATION_PRUNE_FRACTION, MAX_KEY_FACTS - all private. Callers have no way to reason about the health of a memory instance without reimplementing the same arithmetic externally.
What we want
- A pressure signal - the memory instance should expose its current utilization as a reactive value so ChatViewModel can observe it without polling
- A safety buffer - a small portion of the effective budget should be permanently reserved so the user always has room for at least one short final action
- UI awareness - the chat view should show a contextual banner at warning and critical levels with clear actions
Session fork - a "continue in new session" action that seeds the new session with the current structured summary, so context is not lost
Implementation hints
-
TokenAwareSummarizingMemory should expose a StateFlow (e.g. NORMAL / WARNING / CRITICAL) updated after every add(), batch load, and post-summarization. The pressure levels can be simple percentage thresholds against maxTokens - around 70% for warning, 90% for critical.
-
The effective maxTokens should be slightly smaller than the raw 40% allocation - shaving off a few percent as a permanent emergency reserve handles the race window where summarization is in progress and new messages still arrive.
-
ChatState needs a memoryPressureLevel field. ChatViewModel subscribes to the StateFlow from the memory instance and propagates it into state - the same pattern already used for streaming state.
-
The chat view (ChatView.kt) renders a banner above the input area conditioned on pressure level. The WARNING banner is soft and dismissible; the CRITICAL banner is persistent. Both include a "Continue in new session" action.
ChatSessionService needs a forkSessionWithSummary(sourceSessionId) method that creates a new session and seeds its memory with the SessionConversationSummary exported from the source session. The new session starts with no raw messages - only the summary - so it has maximum headroom from the start. The existing exportState() / importState() pair on the memory class is the natural hook for this.
Background
TokenAwareSummarizingMemory manages the context window automatically through background summarization and pruning. This works well under normal conditions. However, the system is entirely silent - users receive no signal about how full a session is, and discover the context limit only when the model returns a hard error and their message is lost.
Problem analysis
No emergency headroom
The memory budget is a hard 40% of the context window. When summarization is already running (summarizationInProgress = true), new messages bypass the threshold check entirely and accumulate unchecked. If the user pastes a large block of code or text during an active summarization cycle, the total can spike past the model's actual limit before the cycle finishes — resulting in a hard model error with no recovery path.
Memory state is completely opaque to the outside world
estimateTotalTokens() is private. There is no way for ChatViewModel, ChatSessionService, or the UI to ask "how full is this session?" The only observable signal is an exception thrown by the model. ChatState has no memory pressure field, so even if the service layer computed pressure, there would be nowhere to put it.
No "continue with context" workflow
When a session does hit its limit, the only option is to manually start a new session from scratch. There is no mechanism to carry the existing SessionConversationSummary - which already exists in memory and in the database - into a new session as a starting point. The structured summary is rich enough to seed a new session meaningfully, but nothing in ChatSessionService does this today.
Threshold constants are invisible to callers
PROTECTED_RECENT_TURNS, SUMMARIZATION_PRUNE_FRACTION, MAX_KEY_FACTS - all private. Callers have no way to reason about the health of a memory instance without reimplementing the same arithmetic externally.
What we want
Session fork - a "continue in new session" action that seeds the new session with the current structured summary, so context is not lost
Implementation hints
TokenAwareSummarizingMemory should expose a StateFlow (e.g. NORMAL / WARNING / CRITICAL) updated after every add(), batch load, and post-summarization. The pressure levels can be simple percentage thresholds against maxTokens - around 70% for warning, 90% for critical.
The effective maxTokens should be slightly smaller than the raw 40% allocation - shaving off a few percent as a permanent emergency reserve handles the race window where summarization is in progress and new messages still arrive.
ChatState needs a memoryPressureLevel field. ChatViewModel subscribes to the StateFlow from the memory instance and propagates it into state - the same pattern already used for streaming state.
The chat view (ChatView.kt) renders a banner above the input area conditioned on pressure level. The WARNING banner is soft and dismissible; the CRITICAL banner is persistent. Both include a "Continue in new session" action.
ChatSessionService needs a forkSessionWithSummary(sourceSessionId) method that creates a new session and seeds its memory with the SessionConversationSummary exported from the source session. The new session starts with no raw messages - only the summary - so it has maximum headroom from the start. The existing exportState() / importState() pair on the memory class is the natural hook for this.