[LLM] Validate accelerator_type for CPU VLLM engine config#64235
[LLM] Validate accelerator_type for CPU VLLM engine config#64235LeThienTrong wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a validation check in VLLMEngineConfig to prevent the use of an accelerator_type with CPU-only configurations, raising an error if both are provided. It also adds corresponding unit tests to verify this behavior. The feedback points out that because VLLMEngineConfig is a Pydantic model, instantiating it with invalid values will raise a pydantic.ValidationError rather than a raw ValueError, and suggests updating the test assertion accordingly.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Signed-off-by: LeThienTrong <lethientrong655@gmail.com>
Signed-off-by: LeThienTrong <lethientrong655@gmail.com>
8e2ee46 to
eb3d361
Compare
|
@LeThienTrong could u run the lint hook? |
Why are these changes needed?
This PR addresses #62138.
LLMConfigalready rejectsaccelerator_typewhen the resolved hardware configuration is CPU-only. However,VLLMEngineConfigcan still be instantiated directly with bothaccelerator_typeandCPUConfig, which can make the accelerator hint inconsistent with a CPU-only configuration.What changes were proposed in this pull request?
VLLMEngineConfigto rejectaccelerator_typewhenaccelerator_configisCPUConfig.accelerator_typeremains valid with a GPU config.How was this patch tested?
python/ray/llm/tests/serve/cpu/configs/test_models.py.I could not run the full test locally because the Ray compiled extension
_rayletis not built in my local Windows source checkout.