Problem
src/routes/proxy.rs fully buffers both the inbound request body and the upstream response body before forwarding:
- Request:
request.into_body().collect().await ... .to_bytes() at :53-63
- Response:
upstream_resp.bytes().await at :99-104
This means:
- Memory usage scales with the largest in-flight request/response, not just concurrency.
- Large uploads/downloads (e.g. file APIs proxied through the gateway) can OOM the process.
- Time-to-first-byte for the client is the full upstream response time, even for streaming endpoints (SSE, chunked downloads).
reqwest is already enabled with the stream feature in Cargo.toml, so the building blocks are in place.
Proposed fix (sketch)
- Forward the inbound body to
reqwest as a stream (reqwest::Body::wrap_stream from the Axum body stream).
- Forward the upstream response body to the client as an Axum
Body built from upstream_resp.bytes_stream().
- Decide whether to keep an upper-bound size limit at the gateway (probably yes, configurable per upstream).
Notes
- Acceptable for MVP under the assumption of small JSON payloads.
- Becomes important once the gateway is used for file uploads, large downloads, or streaming endpoints.
Problem
src/routes/proxy.rsfully buffers both the inbound request body and the upstream response body before forwarding:request.into_body().collect().await ... .to_bytes()at:53-63upstream_resp.bytes().awaitat:99-104This means:
reqwestis already enabled with thestreamfeature inCargo.toml, so the building blocks are in place.Proposed fix (sketch)
reqwestas a stream (reqwest::Body::wrap_streamfrom the Axum body stream).Bodybuilt fromupstream_resp.bytes_stream().Notes