fix(middleware): parse OpenAI-spec tool_choice in /v1/chat/completions#9559
Open
Anai-Guo wants to merge 1 commit intomudler:masterfrom
Open
fix(middleware): parse OpenAI-spec tool_choice in /v1/chat/completions#9559Anai-Guo wants to merge 1 commit intomudler:masterfrom
Anai-Guo wants to merge 1 commit intomudler:masterfrom
Conversation
Follows up on mudler#9526 (the 3-site setter fix) by addressing the remaining clause in mudler#9508 — string mode and OpenAI-spec specific-function shape both silently failed in the /v1/chat/completions parsing path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the fourth-site clause in #9508. This is the natural follow-up to #9526 (merged): #9526 fixed the wrong-setter pattern at three endpoints, and walcz-de's #9509 (merged) fixed the parallel
/v1/responsesparsing. The remaining piece — tool_choice parsing in/v1/chat/completions(SetModelAndConfigmiddleware) — silently mishandled both the string-mode and the OpenAI-spec map shape.The bug
core/http/middleware/request.go:322-335(before this PR) unmarshalled every tool_choice shape throughjson.Unmarshal(..., &functions.Tool{}):Two failure modes, both silenced via
_ =:tool_choice: "required" | "auto" | "none") —json.Unmarshal([]byte("required"), &functions.Tool{})fails.toolChoice.Function.Name == "". The downstream dispatch (lines 487+ in this same file) then runsSetFunctionCallNameString(""), so the requested mode never engages.{"type":"function", "function":{"name":"..."}}) — the JSON keys do not matchfunctions.Tool's field tags (Tool nestsfunctiondifferently), sonameagain ends up empty.The fix
Mirrors the parsing pattern walcz-de established in #9509 for
MergeOpenResponsesConfig, dispatching by the actual incoming shape rather than reinterpreting through the wrong struct:tool_choiceis"required"/"none", hand it toinput.FunctionCallas a string. The existing string-case branch downstream (line 487) routes that throughSetFunctionCallString, which writes the mode field."auto"is the default and skipped. Empty string is skipped.{"type":"function", "function":{"name":"..."}}) and the legacy/Anthropic-compat flat shape ({"type":"function", "name":"..."}). Nested is preferred; flat is the fallback. Setinput.FunctionCall = {"name": <name>}only when a non-empty name was actually found, so the downstream map-case branch writes the name viaSetFunctionCallNameString.This way the existing dispatch on
input.FunctionCall(lines 487-502) keeps doing what it already does correctly — the bug was strictly in the upstream parsing that fed it bad data.The
pkg/functionsimport drops with this change (it was only used for the now-removedvar toolChoice functions.Tool).Net effect (matches what #9509 + #9526 already restored elsewhere)
tool_choice: "required"SetFunctionCallNameString("")(silent no-op)SetFunctionCallString("required")engages modetool_choice: {type:"function", function:{name:"X"}}SetFunctionCallNameString("")(silent no-op)SetFunctionCallNameString("X")engages grammar forcingtool_choice: {type:"function", name:"X"}(legacy)tool_choice: "auto"/"none"""(no-op)Why no new tests in this PR
The parsing logic here is intentionally identical (modulo the string-vs-map dispatch difference dictated by the surrounding code) to
MergeOpenResponsesConfigin #9509, which already has nine Ginkgo specs (MergeOpenResponsesConfig tool_choice parsinginrequest_test.go) covering all four shapes plus malformed inputs. Adding a parallel set forSetModelAndConfigwould require a heavier scaffold (the existing tests there go through Echo + a real model dir) for coverage that's structurally redundant with the merged tests. Happy to add a unit-level test scaffold forSetModelAndConfig's parsing if a maintainer prefers — let me know.Test plan
go build ./...go test ./core/http/middleware/ -count=1 -run TestMiddlewaretool_choice: "required"to/v1/chat/completionswith a tools array → grammar forcing engages, output arrives astool_callsnot free-text.tool_choice: {"type":"function", "function":{"name":"my_tool"}}→my_toolis forced.tool_choice: {"type":"function", "name":"my_tool"}(legacy) → still works.🤖 Generated with Claude Code