Skip to content

perf(trl): set a timeout on vllm client HTTP calls#5288

Open
tejasae-afk wants to merge 1 commit intohuggingface:mainfrom
tejasae-afk:fix/set-a-timeout-on-vllm-client-http-calls
Open

perf(trl): set a timeout on vllm client HTTP calls#5288
tejasae-afk wants to merge 1 commit intohuggingface:mainfrom
tejasae-afk:fix/set-a-timeout-on-vllm-client-http-calls

Conversation

@tejasae-afk
Copy link

@tejasae-afk tejasae-afk commented Mar 14, 2026

Add a timeout to HTTP requests in trl/generation/vllm_client.py to prevent indefinite hangs.

Reason: Network calls without a timeout can hang a worker indefinitely.

Validation: /Users/tejasattarde/Desktop/gh-patchbot/.venv/bin/python -m py_compile trl/generation/vllm_client.py.

Context: py-http-timeout at trl/generation/vllm_client.py:184.


Note

Low Risk
Low risk: only adds a fixed timeout to two requests.get calls to prevent indefinite hangs, without changing request payloads or response handling logic.

Overview
Adds a 10s timeout to vLLM client HTTP health-check and world-size GET requests in trl/generation/vllm_client.py, preventing workers from hanging indefinitely when the server is unreachable or unresponsive.

Written by Cursor Bugbot for commit e76f973. This will update automatically on new commits. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant