DeflateBuffer: tolerate empty chunk in feed_data without crashing on the zlib header sniff#13038
Draft
HrachShah wants to merge 1 commit into
Draft
DeflateBuffer: tolerate empty chunk in feed_data without crashing on the zlib header sniff#13038HrachShah wants to merge 1 commit into
HrachShah wants to merge 1 commit into
Conversation
When a chunked transfer-encoding decoder resumes after a pause, the underlying transport may call DeflateBuffer.feed_data(b"") to give the decoder another chance to make progress. On a stream that has not started decoding yet, feed_data reads chunk[0] to sniff the CM bits of the RFC 1950 zlib header so it can decide whether to enable suppress_deflate_header. With an empty chunk that read raises IndexError, which surfaces to the caller as an unhandled exception during response decoding. Add a guard so the CM-byte sniff is skipped when chunk is empty. The downstream decompressor.decompress_sync(b"") call already handles empty input correctly — it was only the header-sniff branch that crashed.
❌ 2 Tests Failed:
View the top 2 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
Merging this PR will not alter performance
Comparing Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Fixes a crash in
DeflateBuffer.feed_datawhen the chunked transfer-encoding decoder resumes a paused stream by feeding an empty chunk. The CM-byte sniff of the RFC 1950 zlib header was readingchunk[0]on an empty bytes object and raisingIndexErrorto the caller. The fix adds anif chunkguard so the sniff is skipped on empty input, matching the behaviour of the publicStreamReader.feed_data.Why we need it
When a chunked stream is paused (e.g. the consumer is not reading), the underlying transport can call
feed_data(b"")to give the decoder another chance to drain. With an empty chunk, there are no bytes for the zlib-header sniff to inspect, but the sniff was being attempted unconditionally. The resultingIndexError: index out of rangepropagates out as an unhandled exception during response decoding.Repro from the issue:
How I tested it
test_feed_empty_datainTestDeflateBufferexercises the exact repro.IndexError: index out of rangeataiohttp/http_parser.py:1142and passes with the fix.pytest tests/test_http_parser.py -q→ 405 passed, 1 pre-existing failure (test_te_header_non_ascii[py-parser], the test whose comment talks about a non-ASCII Kelvin sign but whose literal text is pure ASCII — the test only passes by happenstance of test order; predates this change).Test log
Checklist
CHANGES/Drafted with Zo Bot; reviewed by HrachShah.