fix(api): generate Mistral-compliant tool-call ids#1997
Open
sam-fakhreddine wants to merge 1 commit into
Open
Conversation
omlx minted tool-call ids as "call_<8 hex>" (13 chars, underscore). Mistral /
tekken (MistralCommonBackend) validates tool_call_id against [a-zA-Z0-9] with a
length of exactly 9 and rejects anything else when the assistant turn is
replayed on the next request -> HTTP 500 on multi-turn tool loops with Devstral,
Mistral-Small, etc. The first turn worked; the second 500'd on the id the server
itself generated.
Centralizes id creation in shared_models.generate_tool_call_id() -> "call" + 5
hex = exactly 9 alphanumeric chars: valid for Mistral and still an opaque id for
OpenAI / Anthropic clients. Replaces all 17 inline call-sites across
tool_calling.py, anthropic_utils.py, responses_utils.py and server.py (dropping
the now-unused uuid import where it was only used for ids). Adds unit tests
asserting the [a-zA-Z0-9]{9} constraint.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1995.
Problem
omlx generates tool-call ids as
call_<8 hex>(e.g.call_49c89af0). Mistral /tekken (
MistralCommonBackend) validatestool_call_idagainst[a-zA-Z0-9]with a length of exactly 9 and rejects anything else when the assistant turn
is replayed on the next request:
So the first turn works and the second 500s — on the id the server itself
generated. Every multi-turn tool loop with Devstral / Mistral-Small breaks.
Fix
Centralize id creation in
shared_models.generate_tool_call_id():Valid for Mistral and still an opaque, recognizable id for OpenAI / Anthropic
clients (which treat the id as opaque — nothing in the codebase matches on the
call_prefix). Replaces all 17 inlinef"call_{uuid.uuid4().hex[:8]}"sites across
tool_calling.py(10),server.py(3),anthropic_utils.py(2)and
responses_utils.py(2), and drops the now-unuseduuidimport where it wasonly used for ids.
Tests
tests/test_shared_models.py::TestGenerateToolCallId— asserts the id is exactly9 chars, matches
[a-zA-Z0-9], has no underscore, and is unique across manydraws.