fix(langchain): serialize tools and ToolMessage metadata in generation input

[filippostanghellini]

Problem

LangChain tools (function definitions passed via invocation_params.tools) and
ToolMessage outputs were not serialized in the generation input, so they did
not appear in the Langfuse UI. The JS SDK diverged from the Python SDK behavior
(__on_llm_action / _convert_message_to_dict), where tools are appended as
{role: "tool"} messages and ToolMessage is serialized with role: "tool" +
tool_call_id.

Changes

• packages/langchain/src/CallbackHandler.ts:
  • handleGenerationStart: read invocation_params.tools and append each as
    {role: "tool", content: tool} to the generation input (matches Python's
    __on_llm_action).
  • extractChatMessageContent: for ToolMessage, set role: "tool" and
    propagate tool_call_id (matches Python's _convert_message_to_dict for
    ToolMessage). Previously role was set to message.name and
    tool_call_id was dropped.
  • Added optional tool_call_id field to LlmMessage / AnonymousLlmMessage
    types.

Verification

• pnpm format:check
• pnpm lint
• pnpm typecheck
• pnpm test (scoped: vitest run --project=unit --project=integration → 389/389 passed; e2e project excluded)
• pnpm test:integration (included in the scoped run above; langchain.integration → 3/3 passed)
• pnpm test:e2e (not run: requires LANGFUSE_BASE_URL, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, OPENAI_API_KEY and a reachable Langfuse server)
• pnpm --filter @langfuse/langchain build

Note: pnpm test:e2e could not run because it requires a reachable Langfuse
server and credentials, which are not available in this environment. The e2e
suite is not affected by this change (no modifications to export protocols,
media upload, or API client).

Release info

Bump level

• Major
• Minor
• Patch

Libraries affected

• All of them
• @langfuse/core
• @langfuse/client
• @langfuse/tracing
• @langfuse/otel
• @langfuse/openai
• @langfuse/langchain

Changelog notes

• Fixed @langfuse/langchain not including tools (invocation_params.tools) and
  ToolMessage metadata (role, tool_call_id) in the generation input,
  aligning behavior with the Python SDK.

Greptile Summary

This patch fixes the LangChain callback handler to include tool definitions (invocation_params.tools) and ToolMessage metadata (role, tool_call_id) in the Langfuse generation input, aligning the JS SDK with Python SDK behavior.

• Tool definitions: handleGenerationStart now reads invocation_params.tools and appends each entry as a {role: "tool", content: tool} message in the generation input.
• ToolMessage: extractChatMessageContent now correctly sets role: "tool" and propagates tool_call_id for tool-type messages; previously role was set to message.name and tool_call_id was dropped.
• Types: LlmMessage and AnonymousLlmMessage gain an optional tool_call_id field, and two integration tests verify both new behaviors end-to-end via span attribute assertions.

Confidence Score: 4/5

The change is narrowly scoped to display/serialization of tool metadata and does not touch any API calls, data persistence, or authentication paths; it is safe to merge.

Both new behaviors are covered by integration tests and the logic is straightforward. The one open question is whether role:"tool" for tool definitions (function schemas from invocation_params.tools) could confuse users in the Langfuse UI, since the same role is also used for tool result messages — there is no runtime breakage, but the ambiguity is worth resolving before the release if trace readability matters.

packages/langchain/src/CallbackHandler.ts — specifically the role assigned to tool definitions appended in handleGenerationStart.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant LC as LangChain
    participant CH as CallbackHandler
    participant LF as Langfuse (OtelSpan)

    LC->>CH: handleChatModelStart(messages, extraParams)
    CH->>CH: extractChatMessageContent per message
    Note over CH: ToolMessage maps to role:tool + tool_call_id
    CH->>CH: handleGenerationStart(prompts, extraParams)
    CH->>CH: read invocation_params.tools
    alt tools present
        CH->>CH: append role:tool + content:toolDef for each tool
    end
    CH->>LF: startAndRegisterOtelSpan with inputMessages

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant LC as LangChain
    participant CH as CallbackHandler
    participant LF as Langfuse (OtelSpan)

    LC->>CH: handleChatModelStart(messages, extraParams)
    CH->>CH: extractChatMessageContent per message
    Note over CH: ToolMessage maps to role:tool + tool_call_id
    CH->>CH: handleGenerationStart(prompts, extraParams)
    CH->>CH: read invocation_params.tools
    alt tools present
        CH->>CH: append role:tool + content:toolDef for each tool
    end
    CH->>LF: startAndRegisterOtelSpan with inputMessages

Loading

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
packages/langchain/src/CallbackHandler.ts:336-344
**`role: "tool"` used for tool definitions, not tool results**

`invocation_params.tools` contains function *definitions* (schemas the model can call), whereas `role: "tool"` conventionally means a tool *result* message (the output of executing a tool). Appending definitions under the same role as `ToolMessage` results makes them indistinguishable in the Langfuse UI — a user looking at the trace cannot tell whether a `role: "tool"` entry describes a callable function or an actual tool output. A more descriptive role like `"function_definition"` or `"tool_definition"` would avoid this ambiguity.

_{Reviews (1): Last reviewed commit: "fix(langchain): serialize tools and Tool..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[vercel]

@filippostanghellini is attempting to deploy a commit to the langfuse Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.