Skip to content

fix(llm): omit temperature for Anthropic Messages protocol - #31

Open
Da-xia-Guo wants to merge 1 commit into
OpenBMB:mainfrom
Da-xia-Guo:fix/anthropic-optional-temperature
Open

fix(llm): omit temperature for Anthropic Messages protocol#31
Da-xia-Guo wants to merge 1 commit into
OpenBMB:mainfrom
Da-xia-Guo:fix/anthropic-optional-temperature

Conversation

@Da-xia-Guo

Copy link
Copy Markdown

Problem

Reasoning models exposed through OpenAI/Anthropic-compatible gateways (e.g. Claude Opus 4.8) reject the temperature parameter outright, returning temperature is deprecated for this model. Because the Anthropic Messages driver sent temperature unconditionally, every request to such a model failed with MODEL_UPSTREAM_ERROR — the model could never pass connection verification (text/stream/json probes) nor be used for chat.

Fix

  • LLMClient now computes request_temperature: None for the anthropic_messages protocol, and unchanged (self.temperature) for all other protocols.
  • Both the non-streaming and streaming request builders use request_temperature.
  • _anthropic_request only adds temperature to the payload when a value is provided (non-None), so Anthropic models that do accept temperature still receive it.

OpenAI Chat Completions and Gemini protocols are unaffected.

Testing

  • Connection verification (text / stream / json probes) against a gateway-hosted claude-opus-4-8 now succeeds (previously failed at the first probe with MODEL_UPSTREAM_ERROR).
  • End-to-end chat turn returns a normal reply.

🤖 Generated with Claude Code

Reasoning models exposed via some OpenAI/Anthropic-compatible gateways
(e.g. Claude Opus 4.8) reject the `temperature` parameter outright
("temperature is deprecated for this model"), so every request failed
with MODEL_UPSTREAM_ERROR.

The Anthropic Messages driver previously sent `temperature`
unconditionally. Now the client omits it for the anthropic_messages
protocol (other protocols unchanged), and the driver only adds
`temperature` to the payload when a value is provided.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@fadeoreo

Copy link
Copy Markdown
Collaborator

I tested this PR against the latest main with:

pytest tests/test_llm_client.py tests/test_anthropic_driver.py -q

The result is 29 failed, 33 passed.

Both generate_text() and generate_text_stream() now access self.request_temperature directly. Existing tests construct lightweight clients with object.__new__(LLMClient) and only initialize temperature, so they fail with:

AttributeError: 'LLMClient' object has no attribute 'request_temperature'

Please preserve compatibility, for example:

getattr(self, "request_temperature", self.temperature)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants