Skip to content

http1connection: accept chunk extensions in chunked request bodies#3677

Open
HrachShah wants to merge 1 commit into
tornadoweb:masterfrom
HrachShah:fix/chunked-body-accept-extensions
Open

http1connection: accept chunk extensions in chunked request bodies#3677
HrachShah wants to merge 1 commit into
tornadoweb:masterfrom
HrachShah:fix/chunked-body-accept-extensions

Conversation

@HrachShah

Copy link
Copy Markdown

A chunked request body line is chunk-size [ chunk-ext ] CRLF per RFC 7230 section 4.1.1, where the optional chunk-ext is one or more ;-separated name[=value] pairs. The pre-fix _read_chunked_body in tornado/http1connection.py passed the entire read line to parse_hex_int, so a client that used any extension (5;ext=val, 5;ext, 5;foo=bar;baz=qux) was rejected with HTTPInputError("invalid chunk size") and a 400. The size was correctly hex; the extension was just appended to it.

The fix takes the size prefix via str.partition(";")[0] before the hex parse. The only genuinely malformed inputs that are rejected now are:

  • an empty chunk header (e.g. just \r\n)
  • a header that starts with ; (no size, only an extension)
  • the existing parse_hex_int failure for non-hex characters

A bare extension with no size (;ext=val) and the leading-; case are covered by the new empty checks. The previous rejection test for 1_a still rejects because parse_hex_int now runs on the size-only substring and 1_a is not all hex.

Two new tests in HTTPServerRawTest:

  • test_chunked_request_body_with_extensions sends 4;ext=val and expects 200 with the full body parsed.
  • test_chunked_request_body_bare_extension_rejected sends ;ext=val and expects 400 with the existing invalid chunk size log line.

Pre-existing behavior preserved: 4\r\nfoo= parses as chunk-size 4, and 1_a is still rejected as non-hex.

Testing

  • python3 -m tornado.test.runtests tornado.test.httpserver_test → 76 tests, OK
  • python3 -m pytest tornado/test/httpserver_test.py -k chunked -p no:aiohttp → 15 passed (was 13, +2 new)
  • The pre-existing test_chunked_request_body_invalid_size still passes (regression check on the 1_a case)

A chunked request body line is 'chunk-size [ chunk-ext ] CRLF' where
chunk-ext is one or more ';'-separated 'name[=value]' pairs. The pre-fix
_read_chunked_body passed the entire read line to parse_hex_int, so a
client that used any extension ('5;ext=val', '5;ext', '5;foo=bar;baz=qux')
got an HTTPInputError('invalid chunk size') and a 400. The size was
correctly hex, the extension was just appended to it.

Take the size prefix via str.partition(';')[0] before the hex parse, and
reject only the genuinely malformed cases:

  * an empty chunk header (e.g. '\r\n' on its own line)
  * a header that starts with ';' (no size, only an extension)
  * the existing parse_hex_int failure for non-hex characters

A bare 'ext' with no size (';ext=val') and the leading ';' case are
covered by the two new empty-checks. The previous '1_a' rejection test
still rejects because parse_hex_int now runs on the size-only substring
and '1_a' is not all hex.

Two new tests in HTTPServerRawTest:
  - test_chunked_request_body_with_extensions sends '4;ext=val' and
    expects 200 with the full body parsed.
  - test_chunked_request_body_bare_extension_rejected sends ';ext=val'
    and expects 400 with the existing 'invalid chunk size' log line.

Pre-existing behavior preserved: '4\r\nfoo=' parses as chunk-size 4, and
'1_a' is still rejected as non-hex.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant