http1connection: accept chunk extensions in chunked request bodies#3677
Open
HrachShah wants to merge 1 commit into
Open
http1connection: accept chunk extensions in chunked request bodies#3677HrachShah wants to merge 1 commit into
HrachShah wants to merge 1 commit into
Conversation
A chunked request body line is 'chunk-size [ chunk-ext ] CRLF' where
chunk-ext is one or more ';'-separated 'name[=value]' pairs. The pre-fix
_read_chunked_body passed the entire read line to parse_hex_int, so a
client that used any extension ('5;ext=val', '5;ext', '5;foo=bar;baz=qux')
got an HTTPInputError('invalid chunk size') and a 400. The size was
correctly hex, the extension was just appended to it.
Take the size prefix via str.partition(';')[0] before the hex parse, and
reject only the genuinely malformed cases:
* an empty chunk header (e.g. '\r\n' on its own line)
* a header that starts with ';' (no size, only an extension)
* the existing parse_hex_int failure for non-hex characters
A bare 'ext' with no size (';ext=val') and the leading ';' case are
covered by the two new empty-checks. The previous '1_a' rejection test
still rejects because parse_hex_int now runs on the size-only substring
and '1_a' is not all hex.
Two new tests in HTTPServerRawTest:
- test_chunked_request_body_with_extensions sends '4;ext=val' and
expects 200 with the full body parsed.
- test_chunked_request_body_bare_extension_rejected sends ';ext=val'
and expects 400 with the existing 'invalid chunk size' log line.
Pre-existing behavior preserved: '4\r\nfoo=' parses as chunk-size 4, and
'1_a' is still rejected as non-hex.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A chunked request body line is
chunk-size [ chunk-ext ] CRLFper RFC 7230 section 4.1.1, where the optionalchunk-extis one or more;-separatedname[=value]pairs. The pre-fix_read_chunked_bodyintornado/http1connection.pypassed the entire read line toparse_hex_int, so a client that used any extension (5;ext=val,5;ext,5;foo=bar;baz=qux) was rejected withHTTPInputError("invalid chunk size")and a 400. The size was correctly hex; the extension was just appended to it.The fix takes the size prefix via
str.partition(";")[0]before the hex parse. The only genuinely malformed inputs that are rejected now are:\r\n);(no size, only an extension)parse_hex_intfailure for non-hex charactersA bare extension with no size (
;ext=val) and the leading-;case are covered by the new empty checks. The previous rejection test for1_astill rejects becauseparse_hex_intnow runs on the size-only substring and1_ais not all hex.Two new tests in
HTTPServerRawTest:test_chunked_request_body_with_extensionssends4;ext=valand expects 200 with the full body parsed.test_chunked_request_body_bare_extension_rejectedsends;ext=valand expects 400 with the existinginvalid chunk sizelog line.Pre-existing behavior preserved:
4\r\nfoo=parses as chunk-size 4, and1_ais still rejected as non-hex.Testing
python3 -m tornado.test.runtests tornado.test.httpserver_test→ 76 tests, OKpython3 -m pytest tornado/test/httpserver_test.py -k chunked -p no:aiohttp→ 15 passed (was 13, +2 new)test_chunked_request_body_invalid_sizestill passes (regression check on the1_acase)