Skip to content

Enable multi-turn messages#618

Open
rebel-jinhwan wants to merge 6 commits intovllm-project:mainfrom
rebel-jinhwan:feat/support-multi-turn
Open

Enable multi-turn messages#618
rebel-jinhwan wants to merge 6 commits intovllm-project:mainfrom
rebel-jinhwan:feat/support-multi-turn

Conversation

@rebel-jinhwan
Copy link

@rebel-jinhwan rebel-jinhwan commented Mar 3, 2026

Summary

This PR adds support for multi-turn chat messages in GuideLLM. When the dataset provides a messages (or conversations / conversation) column, the OpenAI-style request handler uses that column as the full conversation for each request instead of building a single-turn message from prefix_column + text_column + media columns

Details

  • Added "messages_column" to the literal type for dataset column types.
  • Column mapping (src/guidellm/data/preprocessors/mappers.py): messages_column maps from names ["messages", "conversations", "conversation"] so datasets with those column names are recognized.
  • Chat completions formatting (src/guidellm/backends/openai/request_handlers.py): In format(), when building the request body:
    1. Read raw_messages = data.columns.get("messages_column", []).
    2. If raw_messages is a non-empty list and the first element is a list of dicts with role and content, set arguments.body["messages"] = conversation and return.
    3. Otherwise, keep existing behavior: build messages from prefix_column and text_column (and image/video/audio columns) as a single system + user turn.

No changes to scheduler or backend interfaces are required for this; the backend still receives a single request per row, with that request’s body["messages"] already containing the full conversation when messages_column is used.

Test Plan

Related Issues

  • Resolves #

  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

@rebel-jinhwan rebel-jinhwan changed the title enable multi-turn messages Enable multi-turn messages Mar 3, 2026
@rebel-jinhwan rebel-jinhwan force-pushed the feat/support-multi-turn branch from 6a57908 to b6bd9e0 Compare March 3, 2026 16:13
@sjmonson
Copy link
Collaborator

sjmonson commented Mar 5, 2026

Sorry, I'm still mulling over how to best handle this. Ideally we would support some kind of "raw" column which gets merged with the other columns in the formatter. The problem here is that each jsonl row is a raw request so making each row a single key under a new dict level would necessitate changing the dataset format.

Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
@sjmonson sjmonson force-pushed the feat/support-multi-turn branch from 070558d to 1c2016e Compare March 13, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants