Repair gateway's misplaced streamed tool-call id in assembly code#247
Merged
Conversation
The AssemblyAI LLM Gateway's streaming /v1/chat/completions nests the
tool-call `id` inside `function` ({"function": {"id": …, "name": …}})
instead of at the tool-call top level where the OpenAI streaming spec —
and langchain_openai (`id=rtc.get("id")`) — read it. Every streamed tool
call then parses with a name and arguments but `id=None`, so deepagents'
reply ToolMessage fails Pydantic validation (`tool_call_id` must be a
string) and the whole turn errors out:
ValidationError: 1 validation error for ToolMessage
tool_call_id
Input should be a valid string [...input_value=None...]
This made `assembly code` crash on any tool-calling turn (verified live
against the production gateway; the non-streaming endpoint places the id
correctly, so only the streaming path is affected).
Fix it client-side in the existing `_GatewayChatOpenAI` quirk-adapter
(the same subclass that already flattens content arrays the gateway
rejects): override `_convert_chunk_to_generation_chunk` to hoist a
`function.id` back to the tool-call top level before langchain converts
the chunk. The move stays idempotent once the gateway is fixed — a
correct delta puts the id at the top level and leaves no `function.id`,
so it never fires.
The gateway bug is being reported upstream for a server-side fix; this
keeps the CLI resilient regardless of that timeline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
assembly codecrashes on any tool-calling turn with:Root cause (confirmed live against the production gateway)
The AssemblyAI LLM Gateway's streaming
/v1/chat/completionsnests the tool-callidinsidefunctioninstead of at the tool-call's top level, where the OpenAI streaming spec — andlangchain_openai(id=rtc.get("id")) — read it:function.name/function.argumentsare in the right place, so the call assembles with a name and args butid=None. deepagents' ToolNode then buildsToolMessage(tool_call_id=None)→ Pydantic rejects it → the whole turn errors out. The non-streaming endpoint places the id correctly, so only the streaming path (which the agent always uses) is affected.The gateway bug is being reported upstream for a server-side fix; this keeps the CLI resilient regardless of that timeline.
Fix
Client-side, in the existing
_GatewayChatOpenAIquirk-adapter (the same subclass that already flattens content arrays the gateway 500s on): override_convert_chunk_to_generation_chunkto hoist afunction.idback to the tool-call top level before langchain converts the chunk. Split into small rank-A helpers for the complexity gate.Idempotent once the gateway is fixed — a correct delta puts the id at the top level and leaves no
function.id, so the move never fires.Tests
./scripts/check.sh→ All checks passed (100% patch coverage, mutation gate, escape-hatch gate, build+twine all green).🤖 Generated with Claude Code