feat(socket source): add opt-in PROXY protocol v2 support#25908
feat(socket source): add opt-in PROXY protocol v2 support#25908rjshrjndrn wants to merge 9 commits into
Conversation
Add a pure, I/O-free parser for the PROXY protocol v2 binary header: signature check, IPv4/IPv6 address decoding, and a TLV walk that extracts custom 0xE0-0xEF values (e.g. tenant_id) while tolerating unknown TLVs. Covered by unit tests; not yet wired into the source.
Add a fixture captured from haproxy:2.9 with send-proxy-v2 and a custom 0xE0 tenant TLV, confirming the parser's field offsets match real proxy output rather than only self-authored fixtures.
Add an opt-in proxy_protocol flag to the socket TCP source. When enabled, the connection handler reads and strips the v2 header after the TLS handshake and replaces the recorded peer address with the original client address from the header. Gated behind a default-off TcpSource trait method so other TCP sources are unaffected. Verified end-to-end against HAProxy send-proxy-v2: with the flag on the original client IP is recovered and the payload is intact; with it off the raw header bytes leak into the stream.
Surface parsed v2 TLV fields as event metadata under proxy_protocol, with TLV keys as the lowercase hex type byte (e.g. 0xe0) since the wire format identifies fields by numeric type. This lets downstream transforms and sinks read arbitrary proxy-supplied values, for example routing a Kafka topic by a tenant identifier carried in a custom TLV. Verified end-to-end against HAProxy sending two custom TLVs: both appear on the event and drive a templated field value.
Add a changelog fragment, the generated component doc entry for the new proxy_protocol option, and an example config routing a Kafka topic by a tenant id carried in a PROXY protocol v2 custom TLV.
|
All contributors have signed the CLA ✍️ ✅ |
This comment has been minimized.
This comment has been minimized.
|
I have read the CLA Document and I hereby sign the CLA |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b0d7cc5f43
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Address review feedback on the PROXY protocol v2 path: - Guard metadata injection with a log-event check so native metric or trace frames no longer panic the per-connection task. - Bound the header read with the shutdown signal, connection tripwire, and a timeout (max_connection_duration_secs or a short default) so a peer that connects but never sends a header cannot hold a connection-limit permit indefinitely. - Declare the proxy_protocol metadata in the socket source output schema so downstream schema and VRL consumers know the field exists.
Proxies send the PROXY protocol v2 header unencrypted before the TLS handshake (verified against HAProxy send-proxy-v2), so a source that terminates TLS would feed the plaintext header into the acceptor and fail the handshake. Reject the combination at build time with a clear error and document that TLS must be terminated upstream.
The PROXY protocol header deadline was derived from max_connection_duration_secs, so a large value (long-lived connections) reopened the slowloris window. Use a fixed 2s deadline instead: the header is the first bytes after connect and is round-trip-time bound, independent of overall connection lifetime.
The parser ignored the command nibble and extracted addresses whenever the family byte looked valid. A LOCAL header (proxy's own connection, such as a health check) could therefore overwrite host/port with address-looking bytes. Parse the command: for LOCAL, ignore the header addresses and keep the real connection endpoints; for PROXY, parse as before; reject unassigned command values.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 254a0c8cdc
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // header into the TLS acceptor would fail the handshake, so | ||
| // reject the combination explicitly rather than break at | ||
| // runtime. | ||
| if config.proxy_protocol() && config.tls().is_some() { |
There was a problem hiding this comment.
Allow proxy_protocol with explicitly disabled TLS
When a TCP source config contains a tls block with enabled = false, config.tls().is_some() is still true even though MaybeTlsSettings::from_config treats that configuration as a raw, non-TLS listener. In that scenario this rejects an otherwise valid plaintext PROXY-protocol source, so deployments that keep TLS settings present but disabled cannot enable proxy_protocol; check the TLS enable flag or the computed MaybeTlsSettings instead of only the Option.
Useful? React with 👍 / 👎.
| Kind::object(Collection::empty().with_unknown(Kind::bytes())) | ||
| .or_undefined(), |
There was a problem hiding this comment.
Match proxy_protocol schema to emitted metadata
Fresh evidence is that the newly declared kind here still does not match the object built by ProxyHeader::into_metadata, which emits version as an integer and tlvs as a nested object keyed by TLV type. Declaring every top-level proxy_protocol field as bytes makes .proxy_protocol.tlvs look like bytes rather than an object to schema/VRL consumers, including the new routing example that indexes .proxy_protocol.tlvs."0xe0"; model version and the nested tlvs object explicitly instead.
Useful? React with 👍 / 👎.
Summary
Adds opt-in PROXY protocol v2 support to the
socketsource intcpmode viaa new
proxy_protocoloption. When a fronting proxy such as HAProxy terminatesthe client connection and prepends a v2 header (
send-proxy-v2), Vector can now:plaintext hop and a re-encrypted (TLS-bridged) hop share one code path;
header;
proxy_protocol, keyed bythe lowercase hex type byte (for example
proxy_protocol.tlvs."0xe0") sincethe wire format identifies fields by numeric type rather than by name.
This lets operators route on proxy-supplied values, such as a tenant identifier
carried in a custom TLV, without Vector terminating TLS itself.
Parsing is gated behind the explicit opt-in flag (default off) so the
spoofable header is only trusted when a trusted proxy is known to sit in front
of the source.
Scope is intentionally minimal: v2-only, on the
socketsource. PROXYprotocol v1 (ASCII),
syslogwiring, nested SSL sub-TLV CN decoding, and aconfigurable TLV-to-field-name map are deferred to follow-ups.
Vector configuration
Matching HAProxy backend used for testing:
How did you test this PR?
decoding, TLV extraction, tolerant skipping of unknown TLVs, truncation
handling, and the async
read_v2_headerstrip. One test asserts against afixture of real bytes captured from
haproxy:2.9, guarding against aspec-misreading shared by the hand-built fixtures.
haproxy:2.9container configured withsend-proxy-v2and two custom TLVs (0xE0,0xE1) forwarding to a Vectorsocketsource writing to a file sink:proxy_protocol: true, the event carried the original client IP ashost, the payload intact asmessage, and both TLVs underproxy_protocol.tlvs.*;remapderivedtenant_namefromproxy_protocol.tlvs."0xe0"and atemplated field resolved to
logs-rajesh.com;decoded stream and
hostwas the proxy address, confirming the parse pathis only active on opt-in and headers are otherwise treated as payload.
Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.References