Eval bug: SWA on Gemma 4 forgets key details

[frenzybiscuit]

Name and Version

version: b9999
built with GNU 14.3.1 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

7950x + 4x3090

Models

https://huggingface.co/Darkhn-Quants-4/Gemma-4-31B-Animus-V15.0-GGUF/resolve/main/Gemma-4-31B-Animus-V15.1-Experiment-Q8_0-HB16.gguf

Problem description & steps to reproduce

When using SWA, the model tends to forget key details.

Previously, when testing SWA it would randomly insert characters that had left the scene several replies prior, as if they had never left. This has thankfully been fixed in recent llama.cpp builds.

Unfortunately, it -still- forgets key details.

For example, in the below images I am a supernatural entity guiding a soldier, visible only to him. The LLM confirms this in the reply:

Yet, two replies later the other team members can see and hear me.

This does NOT happen with swa-full enabled.

First Bad Commit

No response

Relevant log output

N/A

[am17an]

Can you confirm if some checkpoints are loaded in this whole exchange? You can launch paste the logs with llama-server -lv 4 if possible