test: improve pytest infrastructure and vLLM backend testing#416
Open
planetf1 wants to merge 1 commit intogenerative-computing:mainfrom
Open
test: improve pytest infrastructure and vLLM backend testing#416planetf1 wants to merge 1 commit intogenerative-computing:mainfrom
planetf1 wants to merge 1 commit intogenerative-computing:mainfrom
Conversation
Contributor
|
The PR description has been updated. Please fill out the template for your PR to be reviewed. |
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
planetf1
force-pushed
the
test/vllm-clean
branch
5 times, most recently
from
5a0cdf5 to
d25e4d7
Compare
- Add pytest skip mechanism with capability detection and CLI options - Implement process isolation for GPU-intensive vLLM tests - Enhance test configuration with safe option registration - Fix vLLM structured output token limits and update documentation
planetf1
force-pushed
the
test/vllm-clean
branch
from
d25e4d7 to
cd5a632
Compare
8 tasks
jakelorocco
reviewed
Feb 6, 2026
Contributor
jakelorocco
left a comment
jakelorocco
left a comment
There was a problem hiding this comment.
The changes make sense to me. Can you please provide example outputs of the example and regular tests? I'd like to see what the vllm / heavy_gpu tests running in isolation looks like.
psschwei
reviewed
Feb 6, 2026
| ) | ||
| return backend | ||
| yield backend | ||
| # Cleanup: shutdown vLLM engine and release GPU memory |
Member
There was a problem hiding this comment.
Would it make sense to pull this code into a function that could be used both here and in test_vllm_tools.py rather than repeating it in both places?
| @pytest.fixture(scope="module") | ||
| def backend(): | ||
| """Shared vllm backend for all tests in this module.""" | ||
| # Import cleanup dependencies at top to avoid scoping issues |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improve pytest infrastructure and vLLM backend testing
Type of PR
Description
Fixes #415
Enhances pytest infrastructure with capability detection and process isolation for GPU tests. Fixes process isolation to only activate on CUDA systems, preventing unnecessary overhead on macOS and systems without GPU.
Key changes:
--ignore-gpu-check,--ignore-ram-check, etc.)Testing
Tested
Note that if any large tests are run on cuda, pytest will enforce additional isolation - effectively batching up the test into groups.