Skip to content

Checks for no valid requests in processed dataset#709

Merged
jaredoconnell merged 4 commits intovllm-project:mainfrom
jaredoconnell:fix/dataset-fixes
May 5, 2026
Merged

Checks for no valid requests in processed dataset#709
jaredoconnell merged 4 commits intovllm-project:mainfrom
jaredoconnell:fix/dataset-fixes

Conversation

@jaredoconnell
Copy link
Copy Markdown
Collaborator

@jaredoconnell jaredoconnell commented Apr 28, 2026

Summary

Adds warnings and errors so that problems in the datasets are visible, rather than silently failing.
In main, if the dataset isn't processed correctly, GuideLLM deadlocks while the datasets are pending.

Details

  • Adds failure condition in scheduler/worker_group.py to abort if there are no requests.
  • Adds some other warnings when there are no matching columns

Test Plan

Currently, due to the other bugs, this command triggers the problem:

guidellm benchmark run \
  --target http://localhost:8000/ \
  --data "abisee/cnn_dailymail" \
  --data-args '{"name": "3.0.0"}' \
  --data-column-mapper '{"text_column":"article"}'
  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

@dbutenhof dbutenhof added bug Represents a user-visible defect escape A problem found by an external user in a GuideLLM release. labels Apr 28, 2026
@dbutenhof dbutenhof added this to the v0.7.0 milestone Apr 28, 2026
Comment thread src/guidellm/data/loaders.py Outdated
Comment thread src/guidellm/data/preprocessors/mappers.py
Comment thread src/guidellm/data/loaders.py Outdated
Comment thread src/guidellm/data/loaders.py
Comment thread src/guidellm/scheduler/worker_group.py Outdated
@jaredoconnell jaredoconnell requested a review from sjmonson April 28, 2026 23:08
Comment thread src/guidellm/data/preprocessors/mappers.py Outdated
Comment thread src/guidellm/data/loaders.py
Copy link
Copy Markdown
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my new comments above.

Copy link
Copy Markdown
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concerns are resolved. Leaving the comments open that @dbutenhof added to in-case any of those still need to be addressed.

Comment thread src/guidellm/data/loaders.py
This scenario would previously cause a deadlock on pending.

Assisted-by: Cursor AI
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
@jaredoconnell jaredoconnell merged commit 260915e into vllm-project:main May 5, 2026
11 checks passed
@jaredoconnell jaredoconnell deleted the fix/dataset-fixes branch May 5, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Represents a user-visible defect escape A problem found by an external user in a GuideLLM release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants