Add Responses MVP

[jiahuei]

Summary

This PR adds a full OpenAI Responses API implementation for vLLM. It passes all OpenResponses compatibility tests and ships with a complete MkDocs documentation site and CI pipeline.

What's working

• Responses API backed by Chat Completions (or stateless Responses if desired from core vLLM)
• Verified against all OpenResponses compatibility tests
• Reasoning/thinking pass-through tested with models that support interleaved thinking (GLM, MiniMax)
• 2 built-in tools already working via MCP interface: code interpreter and web search
• Support for remote MCP servers
• Full documentation site with MkDocs + CI pipeline

Why Chat Completions over raw generate?

There has been discussion around bypassing Chat Completions entirely and building the Responses API against vLLM's raw generate (token-in, token-out) endpoint instead. We believe that path introduces significant engineering and maintenance costs:

• Ecosystem fragmentation: Every model support addition and bug fix in core vLLM would need to be duplicated in this project. That's hundreds of existing models, plus every new one going forward.
• Duplicated effort: Chat templating, tokenization, performance metrics, and correctness tests would all need to be re-implemented outside core vLLM. This is work that already exists and is battle-tested upstream.
• Upstream dependency risk: The token-in-token-out endpoint is still WIP. Building on it means getting blocked by its progress, whereas Chat Completions is mature and stable today.

For the one known gap, which is GPT-OSS/Harmony models where Chat Completions may lose channel nuance — the right fix is to ensure Harmony is correctly supported via chat templates and schema extension in core vLLM, not to bypass the entire Chat Completions stack. @bbrowning (Red Hat) agrees: token-to-request conversion logic should live in core vLLM, not be reimplemented externally.

[leseb]

We agreed that the MVP must be a design doc in docs/adr (Architecture Decision Record), to scope the project. Not an actual implementation. An initial implementation will after the first ADR is merged.

Please revisit this. Thanks!

[jiahuei]

Hi @leseb we must have missed this discussion and decision, is there any way we can have more context on it?

[leseb]

Hi @leseb we must have missed this discussion and decision, is there any way we can have more context on it?

Sure, during our sync call last week, the group agreed to proceed with opening a MVP doc as a PR, that will outline the goals, non-goal and provide an initial design (no code) of the project. Here are the community notes: - perhaps the MVP part was unclear, but we verbally said this is a markdown design file, not an actual code implementation :).

[noobHappylife]

Hi @leseb,

Referring to the meeting notes this is a reference design to show how we imagine the implementation would be.

Understood that there's a new discussion on having some RFCs on the design now, We are also preparing some RFCs so we can keep the ball rolling on the MVP implementation details. Thanks :)

[leseb]

Hi @leseb,

Referring to the meeting notes this is a reference design to show how we imagine the implementation would be.

Yes but you don't need code to show how we want the implementation to be :).

Understood that there's a new discussion on having some RFCs on the design now, We are also preparing some RFCs so we can keep the ball rolling on the MVP implementation details. Thanks :)

IMO the RFCs should come first, then the implementation will follow. Otherwise this is backwards.

[tjtanaa]

Agreed.