Skip to content

Misc. bug: webui does not display image attachments returned by MCP tools (tool block regression in #25450, inline references never resolved) #25789

Description

@badbat75

Name and Version

version: b10042 (behavior also verified by reading the tools/ui sources at tag b10042)

Operating systems

All (this is a webui / frontend issue, reproduced on Windows with Chrome)

Which llama.cpp modules do you know to be affected?

llama-server

Command line

llama-server with any tool-calling model, webui with an MCP server enabled whose tool returns image content

Problem description & steps to reproduce

I have an MCP server with a tool that returns a static map as a PNG (standard MCP image content: { type: "image", mimeType: "image/png", data: "<base64>" }). The webui extracts the image correctly: the tool result shows a line like [Attachment saved: mcp-attachment-1752...-1.png] and the image is stored as an IMAGE extra on the tool result message (I can see it in IndexedDB). But the image is never displayed anywhere in the chat. As far as I can tell there are two separate problems:

1. Regression from #25450: the image no longer shows in the expanded tool call block.

Before that PR, ChatMessageAgenticContent.svelte rendered tool results with parseToolResultWithImages(section.toolResult, section.toolResultExtras || message?.extra), so the extras of the tool message were used and the image was visible when expanding the block. After the refactor into ChatMessageToolCallBlock / ChatMessageToolCallBlockDefault, the block receives attachments={message?.extra}, which are the extras of the anchor assistant message and never contain the MCP images. section.toolResultExtras is still populated in deriveSingleTurnSections (tools/ui/src/lib/utils/agentic.ts) but nothing consumes it anymore, so it is dead code now. This used to work, see for example the report in #20314 where the returned image was displayed to the user in the tool block.

2. Older problem: inline references to the attachment never resolve.

When the model references the saved attachment in its answer, for example ![map](mcp-attachment-1752...-1.png) (which is the natural thing for the model to do given the [Attachment saved: ...] marker in the tool output), the reference does not resolve. rehypeResolveAttachmentImages also receives attachments={message?.extra} (in ChatMessageAgenticContent.svelte), so the lookup fails, the img src stays a relative URL, the browser gets a 404 and the user sees the "Image cannot be displayed (open link)" fallback instead of the image. This part predates #25450, I don't think it ever worked.

Steps to reproduce:

  1. Enable an MCP server with a tool that returns image content (a static map tool, an image generation tool, anything).
  2. Ask the model something that triggers the tool.
  3. The tool output contains [Attachment saved: mcp-attachment-...png] but expanding the tool call block shows no image.
  4. If the model embeds the attachment name in its reply, the "Image cannot be displayed (open link)" placeholder is rendered instead of the image.

Possible fix: in ChatMessageAgenticContent.svelte pass a merged attachment pool to the render paths, something like [...(message?.extra ?? []), ...toolMessages.flatMap((m) => m.extra ?? [])]. Attachment names are timestamped so collisions should not be an issue. Or at least restore section.toolResultExtras for the tool call block. Since the extras are already persisted in IndexedDB, existing conversations would start rendering correctly as soon as the fix lands.

First Bad Commit

For the tool block regression: 32beb24 (#25450). The inline reference problem is older and as far as I can tell was never working.

Relevant log output

Browser console shows a 404 for GET /mcp-attachment-<timestamp>-1.png when the assistant message references the attachment name.

Emiliano De Simoni

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions