Docs: troubleshooting guide for XML/tool-call markup leaking into chat sessions (self-hosted llama.cpp / Qwen3 etc.)

[Aaronontheweb]

What the user sees

When chatting with a self-hosted Netclaw connected to a llama.cpp / llama-server backend, the assistant occasionally produces messages or tool calls where raw XML-style markup leaks into visible content. Symptoms include:

• <tool_call>, <function=…>, <parameter=…> tags appearing as plain text in the chat
• Stray </think> (or <think>) tags appearing in the assistant's reply or inside tool-call arguments
• A tool call whose arguments JSON value contains the literal text of another tool call concatenated onto the end
• Tool calls whose argument fields are partially or completely empty (e.g. args={}, {"Path": ""}) when the model clearly intended to populate them
• The same prompt working fine in one session but corrupting in another with longer history

This is almost always a chat-template mismatch on the inference server, not a Netclaw bug. Netclaw faithfully assembles the streaming deltas it receives — if the server emits <tool_call> literal text instead of structured tool-call deltas, that's what Netclaw sees.

Why it happens

Reasoning-capable open-weight models emit tool calls and reasoning blocks with model-specific delimiters. Common patterns:

• Qwen3 family — <tool_call><function=…><parameter=…>…</parameter></tool_call>, reasoning in <think>…</think>
• DeepSeek-R1 family — reasoning in <think>…</think>
• Hermes / Mistral / others — JSON-shaped tool calls

llama-server only knows how to parse these correctly when it's told to use the embedded chat template via --jinja (or an explicit --chat-template-file). Without that, it falls back to a heuristic parser that does not recognize the model's tool-call delimiters and lets the literal markup leak through into streaming output as plain text.

Models known to require --jinja for clean tool calling

This is not exhaustive, but the following families have been reported to exhibit XML / tool-call leakage when run through llama-server without --jinja:

• Qwen3 / Qwen3.5 / Qwen3-Coder — confirmed; <tool_call> XML and </think> leak into content and tool-call args
• Qwen2.5-Instruct (with tool calling) — covered explicitly by llama.cpp's function-calling docs as requiring --jinja
• DeepSeek-R1 distills — reasoning leakage if --reasoning-format is wrong for the consumer

Other reasoning-capable models likely behave similarly. As a rule of thumb, any model whose Hugging Face card describes a tool-call format using XML-style markup or that ships its own chat_template.jinja should be served with --jinja.

Recommended llama-server flags (Qwen3 example)

llama-server \
  --model <path/to/qwen3-gguf> \
  --jinja \
  --reasoning-format deepseek \
  --flash-attn on \
  --ctx-size <N> \
  --parallel <K> \
  --port <P>

• --jinja — mandatory; uses the GGUF's embedded chat template (knows the model's tool-call delimiters)
• --reasoning-format deepseek — correct for Qwen3; the model uses the same <think>/</think> delimiters as DeepSeek
• For some buggy GGUF templates a corrected external template via --chat-template-file <path> is also published by the community (search the model's Hugging Face discussions)

Diagnostic checklist

If a self-hosted Netclaw instance is producing corrupted tool calls or visible XML markup:

1. Check the inference server's launch arguments for --jinja. If absent and the model is Qwen3 / DeepSeek-R1 / similar, that's almost certainly the issue.
2. Check the model card on Hugging Face for the recommended llama-server / vLLM / Ollama command line. If it lists --jinja or a custom chat template file, follow it.
3. Check the llama.cpp build commit for known parser-related regressions if argument fields are empty rather than corrupted with extra text. Check upstream issue/PR history for recent tool-call parser fixes.
4. Try a higher quantization (Q5_K_XL or Q6_K_XL instead of Q4_*). Tool-call structure is documented as quantization-sensitive — sub-4-bit quants frequently produce malformed tool calls even with the right template.
5. Confirm the chat template embedded in the GGUF isn't itself broken — community-corrected templates exist for several Qwen3 quants on Hugging Face.

What Netclaw provides to help diagnose this

Netclaw emits diagnostic counters at three layers around every LLM streaming call:

• SSE layer — what came off the wire from the server (delta counts, suppressed deltas, finish reason)
• Middleware layer — what the chat-client decorator saw before the actor consumed it
• Actor layer — the assembled ChatResponse content breakdown (text chars, thinking chars, tool calls, finish reason)

These show up in the per-session log at ~/.netclaw/logs/sessions/<channel>_<thread>/session.log. If counts match across all three layers but a tool call's arguments field is corrupted, the corruption originates upstream of Netclaw — almost always the inference server's chat template.

What this issue tracks

A short troubleshooting article (FAQ entry or docs page) covering:

• The symptoms above with one or two anonymized example fragments
• A short list of model families known to require --jinja (or equivalent template flag)
• The recommended diagnostic flow when a user reports XML leakage
• A pointer to llama.cpp's function-calling.md and the official Qwen llama.cpp guide

The article shouldn't try to be exhaustive — the goal is to short-circuit the obvious case ("user is on Qwen3 without --jinja") and point the rest at upstream documentation.

References

• llama.cpp docs/function-calling.md
• Qwen llama.cpp guide
• Unsloth Qwen3-Coder template fixes thread — documents the exact failure modes and references community-corrected templates

[Aaronontheweb]

The 3-layer SSE / middleware / actor diagnostic counters described in this issue's "What Netclaw provides to help diagnose this" section are being delivered in netclaw-dev/netclaw#947.

That PR does not close this issue — the docs/troubleshooting article portion (symptoms list, model families requiring --jinja, recommended llama-server flags, diagnostic flow) is still open and tracked here.

[Aaronontheweb]

Additional failure mode: speculative decoding × Qwen3.6

Adding to the symptom catalog this issue covers. Recent debugging on a self-hosted llama.cpp openai-compatible deployment running Qwen3.6 27B Q4 with --jinja --reasoning-format deepseek --spec-default surfaced three distinct failure modes that all resolve to one cause: speculative decoding rejection at chat-template boundaries.

Symptoms observed

1. Mid-list early stop. Model writes 1. <item one content>, starts 2., then emits its own EOS token. finishReason=stop (not length). Output is plain text, just truncated mid-list.
2. Token-repetition loops in sidecar / structured-output calls. The same model, given a JSON-only system prompt (e.g. for memory distillation), enters a state where it generates a token followed by hundreds-to-thousands of repetitions of a 1-2 character suffix (e.g. they'd'd'd'd...). The inference server's parser eventually trips and returns HTTP 500 with Failed to parse input at pos N: <think>....
3. Long SSE silence followed by inactivity-watchdog kill. Streaming calls produce some output, then fall into a similar repetition loop where deltas slow to a trickle and ultimately stop reaching the client. The streaming watchdog eventually trips on the post-prefill inter-delta inactivity timeout.

All three reproduce only when speculative decoding is enabled. Disabling specdec (--spec-default removed; no draft model) eliminates all three without any other config change.

Likely mechanism

Speculative decoding accepts tokens that match between draft and main, rejects on first divergence. The rejection branch lands the verifier on a token position whose logits can be highly skewed by recent context. List markers, end-of-sentence punctuation, and the </think> close are the worst offenders: at those positions the verifier's EOS probability spikes briefly. Specdec rejection there can flip the next token into stop emission (mode 1) or, worse, into a single-token loop with no easy recovery (modes 2 and 3).

Two contributing factors that make this more likely:

• Template mismatch between draft and main models. If --jinja is on the main server but the draft uses a heuristic template, draft and main disagree more often, concentrating rejections at template-state-sensitive positions.
• Q4 quantization. The repetition-loop failure mode is more frequent and more catastrophic on Q4 than Q5/Q6. Q4 quants compress the logit distribution enough that a single high-EOS position can cascade into runaway repetition.

Diagnostic flow that worked

The streaming-call breakdown counters described in the "What Netclaw provides to help diagnose this" section of the issue body landed at Debug level recently in netclaw-dev/netclaw#947. Three layers (SSE wire, middleware, post-assembly) report textDeltas / textChars / thinkingDeltas / thinkingChars / toolCallDeltas / finishReason. With these visible:

• Mid-list truncation reads as textChars=173 finishReason=stop.
• Watchdog hangs read as output=2 finishReason=stop after prolonged silence.
• Repetition loops show up as either inflated thinkingChars with finishReason=length (max-token cap reached) or inflated textChars followed by HTTP 500 from upstream.

Suggested addition to the docs article

When this issue closes the troubleshooting article, a section like the following would cover this class:

Symptom: lists truncate mid-item, single-token repetition loops in tool-call args or structured outputs, or long SSE silences with no error from the server.

Likely cause: speculative decoding paired with a reasoning-format model. Try disabling --spec-default (or removing the draft model) on the inference server. If symptoms disappear, specdec rejection at chat-template boundaries is the cause.

Long-term fix is to switch to MTP (Multi-Token Prediction), which uses the same model's native multi-token heads instead of a draft model. For Qwen3.6, that requires either upstream PR ggml-org/llama.cpp#22673 (still draft) or a working alternative path: AlanHuang99/qwen3.6-mtp-stack (llama.cpp + Vulkan, RDNA4-tested) or vLLM / SGLang with MTP support on ROCm.