Skip to content

feat(socket source): add opt-in PROXY protocol v2 support#25908

Open
rjshrjndrn wants to merge 9 commits into
vectordotdev:masterfrom
rjshrjndrn:master
Open

feat(socket source): add opt-in PROXY protocol v2 support#25908
rjshrjndrn wants to merge 9 commits into
vectordotdev:masterfrom
rjshrjndrn:master

Conversation

@rjshrjndrn

Copy link
Copy Markdown

Summary

Adds opt-in PROXY protocol v2 support to the socket source in tcp mode via
a new proxy_protocol option. When a fronting proxy such as HAProxy terminates
the client connection and prepends a v2 header (send-proxy-v2), Vector can now:

  • parse and strip the header once per connection, after any TLS handshake, so a
    plaintext hop and a re-encrypted (TLS-bridged) hop share one code path;
  • replace the recorded peer address with the original client address from the
    header;
  • expose the v2 TLV fields as event metadata under proxy_protocol, keyed by
    the lowercase hex type byte (for example proxy_protocol.tlvs."0xe0") since
    the wire format identifies fields by numeric type rather than by name.

This lets operators route on proxy-supplied values, such as a tenant identifier
carried in a custom TLV, without Vector terminating TLS itself.

Parsing is gated behind the explicit opt-in flag (default off) so the
spoofable header is only trusted when a trusted proxy is known to sit in front
of the source.

Scope is intentionally minimal: v2-only, on the socket source. PROXY
protocol v1 (ASCII), syslog wiring, nested SSL sub-TLV CN decoding, and a
configurable TLV-to-field-name map are deferred to follow-ups.

Vector configuration

sources:
  tcp_in:
    type: socket
    mode: tcp
    address: "0.0.0.0:9000"
    proxy_protocol: true
    decoding:
      codec: bytes

transforms:
  extract_tenant:
    type: remap
    inputs:
      - tcp_in
    source: |-
      # TLVs are keyed by hex type byte; fall back when the TLV is absent.
      .tenant_name = string(.proxy_protocol.tlvs."0xe0") ?? "unknown"
      .tenant_name = replace(.tenant_name, r'[^a-zA-Z0-9._-]', "_")

sinks:
  kafka_out:
    type: kafka
    inputs:
      - extract_tenant
    bootstrap_servers: "kafka:9092"
    topic: "logs-{{ tenant_name }}"
    encoding:
      codec: json

Matching HAProxy backend used for testing:

backend bk_vector
  mode tcp
  server vector 10.0.0.9:9000 send-proxy-v2 set-proxy-v2-tlv-fmt(0xE0) rajesh.com

How did you test this PR?

  • Unit tests (15) for the parser: signature detection, IPv4/IPv6 address
    decoding, TLV extraction, tolerant skipping of unknown TLVs, truncation
    handling, and the async read_v2_header strip. One test asserts against a
    fixture of real bytes captured from haproxy:2.9, guarding against a
    spec-misreading shared by the hand-built fixtures.
  • End-to-end with a local haproxy:2.9 container configured with
    send-proxy-v2 and two custom TLVs (0xE0, 0xE1) forwarding to a Vector
    socket source writing to a file sink:
    • with proxy_protocol: true, the event carried the original client IP as
      host, the payload intact as message, and both TLVs under
      proxy_protocol.tlvs.*;
    • a remap derived tenant_name from proxy_protocol.tlvs."0xe0" and a
      templated field resolved to logs-rajesh.com;
    • with the flag off (A/B control), the raw header bytes appeared in the
      decoded stream and host was the proxy address, confirming the parse path
      is only active on opt-in and headers are otherwise treated as payload.

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Add a pure, I/O-free parser for the PROXY protocol v2 binary header:
signature check, IPv4/IPv6 address decoding, and a TLV walk that
extracts custom 0xE0-0xEF values (e.g. tenant_id) while tolerating
unknown TLVs. Covered by unit tests; not yet wired into the source.
Add a fixture captured from haproxy:2.9 with send-proxy-v2 and a custom
0xE0 tenant TLV, confirming the parser's field offsets match real
proxy output rather than only self-authored fixtures.
Add an opt-in proxy_protocol flag to the socket TCP source. When
enabled, the connection handler reads and strips the v2 header after
the TLS handshake and replaces the recorded peer address with the
original client address from the header. Gated behind a default-off
TcpSource trait method so other TCP sources are unaffected.

Verified end-to-end against HAProxy send-proxy-v2: with the flag on
the original client IP is recovered and the payload is intact; with it
off the raw header bytes leak into the stream.
Surface parsed v2 TLV fields as event metadata under proxy_protocol,
with TLV keys as the lowercase hex type byte (e.g. 0xe0) since the
wire format identifies fields by numeric type. This lets downstream
transforms and sinks read arbitrary proxy-supplied values, for example
routing a Kafka topic by a tenant identifier carried in a custom TLV.

Verified end-to-end against HAProxy sending two custom TLVs: both
appear on the event and drive a templated field value.
Add a changelog fragment, the generated component doc entry for the
new proxy_protocol option, and an example config routing a Kafka topic
by a tenant id carried in a PROXY protocol v2 custom TLV.
@rjshrjndrn
rjshrjndrn requested review from a team as code owners July 20, 2026 20:22
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@github-actions github-actions Bot added docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: sources Anything related to the Vector's sources domain: external docs Anything related to Vector's external, public documentation and removed docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. labels Jul 20, 2026
@datadog-vectordotdev

This comment has been minimized.

@rjshrjndrn

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b0d7cc5f43

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/sources/util/net/tcp/mod.rs Outdated
Comment thread src/sources/util/net/proxy_protocol.rs Outdated
Comment thread src/sources/util/net/tcp/mod.rs Outdated
Comment thread src/sources/util/net/tcp/mod.rs Outdated
Comment thread src/sources/util/net/tcp/mod.rs Outdated
Address review feedback on the PROXY protocol v2 path:

- Guard metadata injection with a log-event check so native metric or
  trace frames no longer panic the per-connection task.
- Bound the header read with the shutdown signal, connection tripwire,
  and a timeout (max_connection_duration_secs or a short default) so a
  peer that connects but never sends a header cannot hold a
  connection-limit permit indefinitely.
- Declare the proxy_protocol metadata in the socket source output
  schema so downstream schema and VRL consumers know the field exists.
Proxies send the PROXY protocol v2 header unencrypted before the TLS
handshake (verified against HAProxy send-proxy-v2), so a source that
terminates TLS would feed the plaintext header into the acceptor and
fail the handshake. Reject the combination at build time with a clear
error and document that TLS must be terminated upstream.
The PROXY protocol header deadline was derived from
max_connection_duration_secs, so a large value (long-lived
connections) reopened the slowloris window. Use a fixed 2s deadline
instead: the header is the first bytes after connect and is
round-trip-time bound, independent of overall connection lifetime.
@github-actions github-actions Bot added the docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. label Jul 21, 2026
The parser ignored the command nibble and extracted addresses whenever
the family byte looked valid. A LOCAL header (proxy's own connection,
such as a health check) could therefore overwrite host/port with
address-looking bytes. Parse the command: for LOCAL, ignore the header
addresses and keep the real connection endpoints; for PROXY, parse as
before; reject unassigned command values.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 254a0c8cdc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/sources/socket/mod.rs
// header into the TLS acceptor would fail the handshake, so
// reject the combination explicitly rather than break at
// runtime.
if config.proxy_protocol() && config.tls().is_some() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow proxy_protocol with explicitly disabled TLS

When a TCP source config contains a tls block with enabled = false, config.tls().is_some() is still true even though MaybeTlsSettings::from_config treats that configuration as a raw, non-TLS listener. In that scenario this rejects an otherwise valid plaintext PROXY-protocol source, so deployments that keep TLS settings present but disabled cannot enable proxy_protocol; check the TLS enable flag or the computed MaybeTlsSettings instead of only the Option.

Useful? React with 👍 / 👎.

Comment thread src/sources/socket/mod.rs
Comment on lines +264 to +265
Kind::object(Collection::empty().with_unknown(Kind::bytes()))
.or_undefined(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match proxy_protocol schema to emitted metadata

Fresh evidence is that the newly declared kind here still does not match the object built by ProxyHeader::into_metadata, which emits version as an integer and tlvs as a nested object keyed by TLV type. Declaring every top-level proxy_protocol field as bytes makes .proxy_protocol.tlvs look like bytes rather than an object to schema/VRL consumers, including the new routing example that indexes .proxy_protocol.tlvs."0xe0"; model version and the nested tlvs object explicitly instead.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: external docs Anything related to Vector's external, public documentation domain: sources Anything related to the Vector's sources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants