Skip to content

Feature: Replace latency-based connection reuse heuristic with accurate tracking #119

@cbaugus

Description

@cbaugus

Problem

The current connection pool metrics (connection_pool_likely_new_total, connection_pool_likely_reused_total) use a latency heuristic to classify requests as new vs reused connections. Requests slower than metricsReuseThresholdMs (default: 100ms) are counted as "likely new."

This is unreliable because:

  • Fast targets can complete a full TLS handshake under the threshold, making new connections appear reused
  • Slow targets where reused requests exceed the threshold get miscounted as new
  • At high RPS, response latency and handshake latency overlap — no threshold value works correctly
  • The word "likely" in the metric names signals uncertainty to dashboard users

Desired Behavior

Accurate, deterministic reporting of whether each request used a new TCP/TLS connection or reused an existing one. No guessing from latency.

Possible Approaches

  1. Track local socket addresses — Compare the local SocketAddr (ip:port) of each response to previously seen sockets. A new local port = new connection. reqwest 0.12 exposes response.local_addr() which could enable this.

  2. Wrap the connector — Use a custom hyper connector that increments a counter every time a new TCP connection is established, independent of request latency.

  3. Use hyper connection pool events — If hyper exposes pool checkout/miss events via tracing spans, subscribe to them.

Acceptance Criteria

  • Metrics accurately reflect new vs reused connections without relying on latency
  • Remove or deprecate metricsReuseThresholdMs once accurate tracking is in place
  • Metric names should drop the "likely_" prefix (e.g., connection_pool_new_total, connection_pool_reused_total)
  • Works at both low RPS (1 req/5s) and high RPS (10k+)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions