fix(proxy): buffer request body to prevent POST body being dropped upstream

[usernametooshort]

Problem

Fixes #749

When ModifyRequest sets req.ContentLength = -1, Go's HTTP/1.1 transport switches to chunked transfer encoding. In that mode the transport reads req.Body in its own goroutine. Meanwhile, the async logger (pkg/logger/logger.go) also reads the same req.Body pointer via httputil.DumpRequest. Whichever goroutine wins the race drains the body; the other gets an empty reader.

The result: upstream servers receive Transfer-Encoding: chunked with a zero-length body, while Proxify's own log shows the body correctly (log wins the race, transport loses).

Root Cause

// ModifyRequest — before this fix
req.ContentLength = -1   // forced chunked; body race with async logger

Fix

Buffer the entire request body in ModifyRequest before it is handed to either the logger or the transport:

if req.Body != nil && req.Body != http.NoBody {
    bodyBytes, err := io.ReadAll(req.Body)
    _ = req.Body.Close()
    if err == nil {
        req.Body = io.NopCloser(bytes.NewReader(bodyBytes))
        req.ContentLength = int64(len(bodyBytes))
    }
}

This ensures:

1. Both the logger and the transport read from the same in-memory buffer (no race)
2. The forwarded request uses Content-Length (not chunked), matching what the upstream originally expected
3. match-replace DSL operations are unaffected

Verification

Using the reproduction steps from #749:

# Terminal 1 — simple upstream server
python3 -c "
import sys
from http.server import HTTPServer, BaseHTTPRequestHandler
class H(BaseHTTPRequestHandler):
    def do_POST(self):
        n = int(self.headers.get(str(chr(67))+str(chr(111))+str(chr(110))+str(chr(116))+str(chr(101))+str(chr(110))+str(chr(116))+str(chr(45))+str(chr(76))+str(chr(101))+str(chr(110))+str(chr(103))+str(chr(116))+str(chr(104)), 0))
        b = self.rfile.read(n)
        print(f'body={b}')
        self.send_response(200); self.end_headers()
HTTPServer(('\', 9999), H).serve_forever()
"
# Terminal 2 — proxify
proxify -http-addr 127.0.0.1:8888
# Terminal 3
curl -x http://127.0.0.1:8888 -X POST -d "username=admin&password=secret" http://127.0.0.1:9999/login
# Expected: body=b'username=admin&password=secret'
# Before fix: body=b''  (empty)

[neo-by-projectdiscovery-dev]

Neo - PR Security Review

Critical: 1

Highlights

• Fixes race condition between async logger and HTTP transport that caused POST bodies to be dropped upstream
• Introduces request body buffering to ensure both logger and transport can read the full body
• Changes from chunked transfer encoding (Content-Length: -1) to explicit Content-Length header

Critical (1)

• Unbounded memory allocation enables memory exhaustion DoS — proxy.go:217
  The ModifyRequest function calls io.ReadAll(req.Body) without any size limit, loading the entire request body into memory. An attacker can send arbitrarily large POST bodies (gigabytes) to exhaust server memory and cause denial of service.

Security Impact

Unbounded memory allocation enables memory exhaustion DoS (proxy.go:217):
Attacker can exhaust proxy memory by sending multiple concurrent requests with multi-gigabyte POST bodies, causing OOM kills and service disruption. No authentication or special access is required - any client connecting to the proxy can trigger this.

Attack Examples

Unbounded memory allocation enables memory exhaustion DoS (proxy.go:217):

curl -x http://proxy:8888 -X POST -H 'Content-Length: 5000000000' --data-binary @5GB_file http://example.com

Or send 100 concurrent 100MB requests to exhaust ~10GB RAM:
for i in {1..100}; do (curl -x http://proxy:8888 -X POST -d "$(head -c 100M /dev/zero | base64)" http://example.com &); done

Suggested Fixes

Unbounded memory allocation enables memory exhaustion DoS (proxy.go:217):

Wrap req.Body with io.LimitReader before calling io.ReadAll to enforce a maximum body size. Example: bodyBytes, err := io.ReadAll(io.LimitReader(req.Body, 10*1024*1024)) // 10MB limit. Consider making the limit configurable.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@proxy.go` at line 217, the code calls io.ReadAll(req.Body) without any size
restriction; wrap req.Body with io.LimitReader(req.Body, maxBodySize) before
reading to prevent unbounded memory allocation. Add a configurable maxBodySize
field to Options (e.g., default 100MB) and reject requests exceeding this limit
with HTTP 413 Request Entity Too Large.

Hardening Notes

• Add configurable maximum body size limit (RequestBodySizeLimit option) with sensible default like 100MB to balance functionality with resource protection
• Consider adding metrics/logging for rejected oversized requests to detect abuse patterns
• Document the body size limit in user-facing documentation so operators can tune it based on their use case

_{Comment @pdneo help for available commands. · Open in Neo}

[neo-by-projectdiscovery-dev]

🔴 Unbounded memory allocation enables memory exhaustion DoS (CWE-400) — The ModifyRequest function calls io.ReadAll(req.Body) without any size limit, loading the entire request body into memory. An attacker can send arbitrarily large POST bodies (gigabytes) to exhaust server memory and cause denial of service.

Attack Example

curl -x http://proxy:8888 -X POST -H 'Content-Length: 5000000000' --data-binary @5GB_file http://example.com

Or send 100 concurrent 100MB requests to exhaust ~10GB RAM:
for i in {1..100}; do (curl -x http://proxy:8888 -X POST -d "$(head -c 100M /dev/zero | base64)" http://example.com &); done

Suggested Fix

Wrap req.Body with io.LimitReader before calling io.ReadAll to enforce a maximum body size. Example: bodyBytes, err := io.ReadAll(io.LimitReader(req.Body, 10*1024*1024)) // 10MB limit. Consider making the limit configurable.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@proxy.go` at line 217, the code calls io.ReadAll(req.Body) without any size
restriction; wrap req.Body with io.LimitReader(req.Body, maxBodySize) before
reading to prevent unbounded memory allocation. Add a configurable maxBodySize
field to Options (e.g., default 100MB) and reject requests exceeding this limit
with HTTP 413 Request Entity Too Large.