fix: disable nginx proxy buffering to enable AI streaming responses#1600
Conversation
Nginx was buffering the entire backend response before forwarding it to the client, preventing SSE streaming on the /ai/v4/request endpoint. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes a critical issue preventing Server-Sent Events (SSE) streaming on AI endpoints by configuring nginx to disable response buffering. The backend uses SSE to stream AI-generated responses incrementally to clients, but nginx was buffering the entire response before forwarding it, defeating the purpose of streaming.
Changes:
- Disabled nginx proxy buffering and caching for the
/api/location block - Added HTTP/1.1 protocol support and proper Connection header handling for persistent connections
- Fixed indentation/formatting inconsistencies (tabs to spaces)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| proxy_set_header Host $host; | ||
| proxy_http_version 1.1; | ||
| proxy_set_header Connection ''; | ||
| proxy_buffering off; |
There was a problem hiding this comment.
Consider adding proxy_read_timeout configuration to prevent premature connection timeouts during long-running AI streaming responses. SSE connections can remain open for extended periods, and the default nginx timeout (60s) may cause issues with longer AI processing. Add something like proxy_read_timeout 300s; to allow sufficient time for AI responses.
| proxy_buffering off; | |
| proxy_buffering off; | |
| proxy_read_timeout 300s; |
| proxy_pass http://localhost:3000/; | ||
| proxy_set_header Host $host; | ||
| proxy_http_version 1.1; | ||
| proxy_set_header Connection ''; |
There was a problem hiding this comment.
Consider adding proxy_set_header X-Accel-Buffering no; to explicitly disable nginx's internal buffering mechanism. While proxy_buffering off; disables the main buffering, X-Accel-Buffering provides an additional layer of control specifically designed for streaming responses and ensures no buffering occurs at any level in nginx.
| proxy_set_header Connection ''; | |
| proxy_set_header Connection ''; | |
| proxy_set_header X-Accel-Buffering no; |
Nginx was buffering the entire backend response before forwarding it to the client, preventing SSE streaming on the /ai/v4/request endpoint.