Conversation
The DTLS 1.3 auto-sense server previously fell back to DTLS 1.2 on any ParseError or ParseIncomplete during AwaitClientHello. That meant a single corrupted fragment of a real DTLS 1.3 ClientHello, or a stray non-handshake packet from off-path traffic, could force a downgrade. Gate the parse-error fallback on a lightweight structural check: fall back only when the packet at least claims to be a Handshake record carrying a ClientHello message. Random/garbage packets still bubble the parse error up and the server stays in 1.3 auto-sense. The check runs unconditionally before matching on the parser result so the time spent in the auto-sense dispatch does not depend on which error branch was taken. The clean Dtls12Fallback path (supported_versions parsed but did not include 1.3) is unchanged.
Extract client_hello_handshake() so looks_like_client_hello and client_hello_wants_psk both go through the same record-header + handshake-header validation. Pure dedup, no behavior change.
algesten
force-pushed
the
fix/auto-sense-fallback-discriminate-parse-errors
branch
from
bcf80b7 to
0044ddf
Compare
The previous structural check accepted any record with content_type=22 and a 12-byte handshake header starting with msg_type=ClientHello — including a header-only fake with declared length=0. That kept the DTLS 1.2 fallback predicate too loose: a tiny crafted packet could still trigger fallback even though no real ClientHello followed. Tighten the check to also require: - fragment_offset + fragment_length <= length (no fragment overflows the declared total) - 12 + fragment_length <= record_body.len() (the fragment bytes declared are actually present in the record) - For an unfragmented CH (frag_off == 0 && frag_len == length), length must be >= 41 — the minimum byte count any real DTLS 1.2 ClientHello can carry (version + random + sid_len + cookie_len + suites_len + comp_len + comp). Fragmented first/middle fragments still pass since the size floor only applies to single-record CHs. Replace the test that asserted a header-only CH passes with one that uses a minimum-shape body. Add negative tests for: header-only CH, undersized unfragmented CH, fragment-offset/length overflow, missing fragment bytes. Add positive tests for first-fragment and non-first- fragment of a fragmented CH so the legitimate fragmented path stays covered.
Tighten looks_like_client_hello to also require fragment_offset == 0. A non-first fragment arriving alone could be a spoofed packet aimed at forcing a downgrade — real fragmented ClientHellos always include a frag_off=0 fragment, and the clean Dtls12Fallback path (driven by supported_versions, not by this gate) handles fully reassembled fragmented 1.2 CHs once they complete. Replace the looks_like_client_hello_accepts_non_first_fragment test with looks_like_client_hello_rejects_non_first_fragment. Add auto_server_falls_back_on_ch_shaped_malformed_packet — an integration test that documents the intentional behavior of the gated fallback: a packet that is structurally a ClientHello but whose body the DTLS 1.3 engine cannot parse should still flip the auto-sense server into DTLS 1.2 mode. (Pairs with auto_server_drops_garbage_without_falling_back, which verifies that random non-CH garbage does not.)
xnorpx
approved these changes
Apr 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Dtls::handle_pending_autofell back to DTLS 1.2 on anyParseError/ParseIncompletereturned by the 1.3 engine duringAwaitClientHello. A single corrupted fragment of a real 1.3 ClientHello, or stray non-handshake traffic, could force a downgrade.looks_like_client_hello(packet)— a lightweight structural check (Handshake record carrying msg_type=ClientHello) — and gate the parse-error fallback on it. Garbage packets bubble the error up; the server stays in 1.3 auto-sense.Dtls12Fallbackpath (supported_versions parsed but did not include 1.3) is unchanged — that is the intended downgrade trigger for real 1.2 clients.Test plan
looks_like_client_hellocovering: valid CH record, non-handshake content type, other handshake message types (ServerHello, HVR, Certificate, Finished), truncated packets, short handshake body.auto_server_drops_garbage_without_falling_backtest confirms a garbage first packet does not flip the auto-sense server to Server12.tests/auto/*integration tests pass (62/62), including fragmented-CH and PSK-fallback scenarios.