fix(model-router): bound SSE rewriter holdback buffer#1933
Open
Lightheartdevs wants to merge 1 commit into
Open
fix(model-router): bound SSE rewriter holdback buffer#1933Lightheartdevs wants to merge 1 commit into
Lightheartdevs wants to merge 1 commit into
Conversation
_SSERewriter.buffer accumulated every chunk but only trimmed when an SSE delimiter matched, so an upstream that streamed without a delimiter grew it without limit (MAX_BODY_BYTES only guards the request body). Bound the held-back partial event at MAX_SSE_HOLDBACK (1MB, env-overridable): once exceeded, flush it via _rewrite_sse_event (same best-effort rewrite as finish()) and reset. Fix-forward carry-over from the #1914/#1931 review (mirrors the 1MB holdback bound noted in #1920). Adds test_sse_holdback_is_bounded_without_delimiter; full router suite (39 tests) green, ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lightheartdevs
force-pushed
the
fix/sse-rewriter-bounded-holdback
branch
from
July 21, 2026 08:28
a74d232 to
7f9a499
Compare
Collaborator
Author
|
Thanks for taking on the model-router streaming edge case; bounding the SSE rewriter holdback is a useful hardening step for malformed or delimiter-less upstream streams. Advisory status: looks merge-ready What looks good:
What I checked:
Thanks again for pushing this forward. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix-forward carry-over from #1914/#1931 review
_SSERewriter.bufferappended every streamed chunk but only trimmed it when an SSE delimiter matched. An upstream that streams without a delimiter grew the buffer without bound —MAX_BODY_BYTESonly guards the inbound request body, not the rewriter's held-back partial event. (Flagged in the #1914 Claude-lane review; #1920 had bounded its holdback at 1MB.)Change
MAX_SSE_HOLDBACK(default 1MB,ODS_ROUTER_MAX_SSE_HOLDBACK_BYTES)._SSERewriter.feed()flushes the partial buffer via_rewrite_sse_event(same best-effort rewritefinish()does) once it exceeds the cap, then resets. Memory bounded; framing continues correctly afterward.Test
test_sse_holdback_is_bounded_without_delimiter: a delimiter-less oversized chunk is flushed (not held), buffer stays ≤ cap, subsequent well-framed event still frames.🤖 Generated with Claude Code