Skip to content

Repair gateway's misplaced streamed tool-call id in assembly code#247

Merged
alexkroman merged 2 commits into
mainfrom
fix-gateway-streaming-tool-call-id
Jun 18, 2026
Merged

Repair gateway's misplaced streamed tool-call id in assembly code#247
alexkroman merged 2 commits into
mainfrom
fix-gateway-streaming-tool-call-id

Conversation

@alexkroman

Copy link
Copy Markdown
Collaborator

Problem

assembly code crashes on any tool-calling turn with:

✗ ValidationError: 1 validation error for ToolMessage
tool_call_id
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]

Root cause (confirmed live against the production gateway)

The AssemblyAI LLM Gateway's streaming /v1/chat/completions nests the tool-call id inside function instead of at the tool-call's top level, where the OpenAI streaming spec — and langchain_openai (id=rtc.get("id")) — read it:

// actual (streaming first delta)
"tool_calls": [{ "index": 0, "function": { "id": "toolu_…", "name": "get_weather", "arguments": "" } }]

// expected (OpenAI spec)
"tool_calls": [{ "index": 0, "id": "toolu_…", "type": "function", "function": { "name": "get_weather", "arguments": "" } }]

function.name/function.arguments are in the right place, so the call assembles with a name and args but id=None. deepagents' ToolNode then builds ToolMessage(tool_call_id=None) → Pydantic rejects it → the whole turn errors out. The non-streaming endpoint places the id correctly, so only the streaming path (which the agent always uses) is affected.

The gateway bug is being reported upstream for a server-side fix; this keeps the CLI resilient regardless of that timeline.

Fix

Client-side, in the existing _GatewayChatOpenAI quirk-adapter (the same subclass that already flattens content arrays the gateway 500s on): override _convert_chunk_to_generation_chunk to hoist a function.id back to the tool-call top level before langchain converts the chunk. Split into small rank-A helpers for the complexity gate.

Idempotent once the gateway is fixed — a correct delta puts the id at the top level and leaves no function.id, so the move never fires.

Tests

  • branch-by-branch coverage of the hoist logic (malformed variants skipped, only a function-nested id is hoisted)
  • guard early-returns
  • end-to-end: a real gateway-shaped chunk run through the override yields a tool call with its id intact

./scripts/check.shAll checks passed (100% patch coverage, mutation gate, escape-hatch gate, build+twine all green).

🤖 Generated with Claude Code

The AssemblyAI LLM Gateway's streaming /v1/chat/completions nests the
tool-call `id` inside `function` ({"function": {"id": …, "name": …}})
instead of at the tool-call top level where the OpenAI streaming spec —
and langchain_openai (`id=rtc.get("id")`) — read it. Every streamed tool
call then parses with a name and arguments but `id=None`, so deepagents'
reply ToolMessage fails Pydantic validation (`tool_call_id` must be a
string) and the whole turn errors out:

    ValidationError: 1 validation error for ToolMessage
    tool_call_id
      Input should be a valid string [...input_value=None...]

This made `assembly code` crash on any tool-calling turn (verified live
against the production gateway; the non-streaming endpoint places the id
correctly, so only the streaming path is affected).

Fix it client-side in the existing `_GatewayChatOpenAI` quirk-adapter
(the same subclass that already flattens content arrays the gateway
rejects): override `_convert_chunk_to_generation_chunk` to hoist a
`function.id` back to the tool-call top level before langchain converts
the chunk. The move stays idempotent once the gateway is fixed — a
correct delta puts the id at the top level and leaves no `function.id`,
so it never fires.

The gateway bug is being reported upstream for a server-side fix; this
keeps the CLI resilient regardless of that timeline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alexkroman alexkroman enabled auto-merge June 18, 2026 19:26
@alexkroman alexkroman added this pull request to the merge queue Jun 18, 2026
Merged via the queue into main with commit 0eb10f7 Jun 18, 2026
20 checks passed
@alexkroman alexkroman deleted the fix-gateway-streaming-tool-call-id branch June 18, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants