fix(llm): omit temperature for Anthropic Messages protocol - #31
Open
Da-xia-Guo wants to merge 1 commit into
Open
fix(llm): omit temperature for Anthropic Messages protocol#31Da-xia-Guo wants to merge 1 commit into
Da-xia-Guo wants to merge 1 commit into
Conversation
Reasoning models exposed via some OpenAI/Anthropic-compatible gateways
(e.g. Claude Opus 4.8) reject the `temperature` parameter outright
("temperature is deprecated for this model"), so every request failed
with MODEL_UPSTREAM_ERROR.
The Anthropic Messages driver previously sent `temperature`
unconditionally. Now the client omits it for the anthropic_messages
protocol (other protocols unchanged), and the driver only adds
`temperature` to the payload when a value is provided.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collaborator
|
I tested this PR against the latest main with:
The result is Both
Please preserve compatibility, for example: getattr(self, "request_temperature", self.temperature) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Reasoning models exposed through OpenAI/Anthropic-compatible gateways (e.g. Claude Opus 4.8) reject the
temperatureparameter outright, returningtemperature is deprecated for this model. Because the Anthropic Messages driver senttemperatureunconditionally, every request to such a model failed withMODEL_UPSTREAM_ERROR— the model could never pass connection verification (text/stream/json probes) nor be used for chat.Fix
LLMClientnow computesrequest_temperature:Nonefor theanthropic_messagesprotocol, and unchanged (self.temperature) for all other protocols.request_temperature._anthropic_requestonly addstemperatureto the payload when a value is provided (non-None), so Anthropic models that do accepttemperaturestill receive it.OpenAI Chat Completions and Gemini protocols are unaffected.
Testing
claude-opus-4-8now succeeds (previously failed at the first probe withMODEL_UPSTREAM_ERROR).🤖 Generated with Claude Code