Skip to content

perf: stream proxy request/response bodies instead of buffering #67

Description

@ttatsato

Problem

src/routes/proxy.rs fully buffers both the inbound request body and the upstream response body before forwarding:

  • Request: request.into_body().collect().await ... .to_bytes() at :53-63
  • Response: upstream_resp.bytes().await at :99-104

This means:

  • Memory usage scales with the largest in-flight request/response, not just concurrency.
  • Large uploads/downloads (e.g. file APIs proxied through the gateway) can OOM the process.
  • Time-to-first-byte for the client is the full upstream response time, even for streaming endpoints (SSE, chunked downloads).

reqwest is already enabled with the stream feature in Cargo.toml, so the building blocks are in place.

Proposed fix (sketch)

  • Forward the inbound body to reqwest as a stream (reqwest::Body::wrap_stream from the Axum body stream).
  • Forward the upstream response body to the client as an Axum Body built from upstream_resp.bytes_stream().
  • Decide whether to keep an upper-bound size limit at the gateway (probably yes, configurable per upstream).

Notes

  • Acceptable for MVP under the assumption of small JSON payloads.
  • Becomes important once the gateway is used for file uploads, large downloads, or streaming endpoints.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions