Eval bug: GGML_ASSERT "tensor write out of bounds" in server_prompt_cache::load with Qwen3.6-27B (Vulkan)

[crxtasy12]

Name and Version

llama-server build b9754
Linux (Pop!_OS 24.04 LTS, kernel 7.0.11-76070011-generic

Operating systems

Linux

GGML backends

Vulkan

Hardware

• CPU: AMD Ryzen 7 1700
• GPU: Intel Arc Pro B70 (Battlemage, BMG G31), 32GB VRAM
• Vulkan device reported: Vulkan0 : Intel(R) Graphics (BMG G31)

Models

Qwen3.6-27B-Q4_K_M (dense, not the MoE variant)

Problem description & steps to reproduce

Problem description & steps to reproduce

Crash occurred during a long multi-turn agentic conversation (~32k tokens of accumulated context) driven by a custom agent client making repeated chat-completion requests. The crash happened on a request that matched a previously cached prompt at very high but not exact similarity:

found better prompt with f_keep = 0.996, sim = 0.995

This triggered context-checkpoint restoration, which hit an out-of-bounds tensor write and aborted the process (SIGABRT, core dump). Restart=on-failure under systemd recovered the service automatically about a minute later, but the in-flight client request failed.

First Bad Commit

Unknown — first observed on build b9754.

Relevant log output

llama-server -m Qwen3.6-27B-Q4_K_M.gguf
--device Vulkan0 -ngl 999 --ctx-size 65536 --jinja
--reasoning off --no-mmproj --host 0.0.0.0 --port 8080

(prompt cache and context checkpoints enabled by default per #16391 — size limit 8192 MiB, max 32 checkpoints)

found better prompt with f_keep = 0.996, sim = 0.995
/home/runner/work/llama.cpp/llama.cpp/ggml/src/ggml-backend.cpp:334: GGML_ASSERT(offset + size <= ggml_nbytes(tensor) && "tensor write out of bounds") failed

libggml-base.so.0(ggml_print_backtrace+0x21f)
libggml-base.so.0(ggml_abort+0x152)
libggml-base.so.0(ggml_backend_tensor_set+0x109)
libllama.so.0(llama_io_read_host::~llama_io_read_host()+0x49)
libllama.so.0(llama_context::state_seq_set_data(int, unsigned char const*, unsigned long, unsigned int)+0x1ce)
libllama-server-impl.so(server_prompt_cache::load(server_prompt&, server_tokens const&, llama_context*, llama_context*, int)+0x1ea)
libllama-server-impl.so(server_context_impl::get_available_slot(server_task const&)+0x218)
libllama-server-impl.so(server_context_impl::process_single_task(server_task&&)+0x58b)
libllama-server-impl.so(server_queue::start_loop(long)+0x86)
libllama-server-impl.so(llama_server(int, char**)+0x2ebb)

systemd: Main process exited, code=dumped, status=6/ABRT

Possibly related

• Eval bug: llama-server crashes on CUDA with Qwen3.6-27B #23210 — same model (Qwen3.6-27B), same feature area (context-checkpoint restore), very similar similarity values (sim_best=0.843/f_keep=0.990 there vs sim=0.995/f_keep=0.996 here), but a different backend (CUDA) and a generic CUDA error rather than this specific assert — may be the same underlying bug surfacing differently per backend.
• Eval bug: Qwen 3.6 27B forcing full prompt re-processing due to lack of cache data #22746 — Qwen 3.6 27B forcing full prompt reprocessing due to lost cache data — likely the non-fatal version of the same fragility (my own log shows this exact warning on a later request that didn't crash).
• kv-cache : add SWA support #13194 (PR discussion) — a maintainer/contributor noted Qwen 3.6 series "consistently fail to reuse cache" while Gemma 4 works properly with this same feature, suggesting this is a Qwen-3.6-specific weak point in the SWA/checkpoint cache path rather than a generic bug.

[ggerganov]

Do you have a consistent way to reproduce this?