fix(ds4): preserve exact speculative verification

[cheese-cakee]

Summary

• preserve every verifier logits row when DS4 speculative verification requests all_logits_out, instead of applying the multi-token last-row output optimization
• make DFLASH_DS4_FUSED_VERIFY=1 fail closed to the normal verifier because the persistent fused graph is not token-exact at near-tied logits
• retain the approximate fused graph only behind the additional, explicitly named DFLASH_DS4_ALLOW_APPROX_FUSED_VERIFY=1 research opt-in
• add default-off token/logit parity diagnostics, a deterministic adaptive-width override, and a production-model regression script
• document the exact/approximate verification boundary and the new controls

Root cause

Current main combines the multi-token output-projection optimization with a verifier hook that still consumes one logits row per verified token. The projection could allocate only the final row while the verifier read all q rows, triggering GGML_ASSERT(offset + size <= ggml_nbytes(tensor)) in normal DSpark verification.

After restoring the requested row count, the fused verifier still diverged deterministically. Its persistent whole-model HIP graph uses a different HC/reduction topology from the normal causal verifier. Identical inputs, positions, and cache state therefore produce small numerical differences that can flip a near-tied greedy token; this is a numerical-parity boundary, not a binding or cache-state bug.

Runtime behavior

DFLASH_DS4_FUSED_VERIFY=1 now emits a warning and uses the normal verifier. The previous graph remains available only when both the public flag and DFLASH_DS4_ALLOW_APPROX_FUSED_VERIFY=1 are set, making approximate inference an explicit research choice.

DFLASH_DS4_PARITY_TRACE=1 records full generated token IDs plus per-row top-two logits and margins. DFLASH_DS4_ADAPTIVE_WIDTH=0 fixes the verification width for deterministic comparisons and takes precedence over the legacy /tmp/ds4_awidth control. Both diagnostics are off by default.

Reproduction

Hardware and software:

• Strix Halo gfx1151, ROCm 7.2.4
• current main: 405ab02482592f248f8f574087cf6964e06698b7
• target: DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf, SHA-256 efc7ed607ff27076e3e501fc3fefefa33c0ed8cf1eff483a2b7fdc0c2e616668
• draft: ds4-dspark-draft.gguf, SHA-256 48883d35b8a67ecfd2858a90e12a47d04cb5ac581acef868ca0f58544816f746
• prompt: Explain why a bicycle stays upright while moving.
• temperature 0, seed 1234, fixed q=4, adaptive width disabled, model-default six experts

The first divergence occurs on the eighth generated token:

Verifier	Selected token	Selected logit	Competing token/logit	Margin
normal	270	37.9972153	436 / 37.9925690	0.0046463
approximate fused	436	38.2708359	270 / 37.6983910	0.5724449

Tracing confirmed identical verifier inputs, positions, and cache state before the first fused batch. The approximate graph already shows numerical differences at the initial verification step.

Validation

• fresh ROCm Release build for gfx1151: passed
• test_deepseek4_unit: passed
• production dflash_server model-backed regression: PASS: 32 generated token IDs are identical
• explicit approximate opt-in: reproduced the historical token divergence
• python -m py_compile server/scripts/test_ds4_fused_verify_parity.py: passed
• git diff --check: passed

The model-backed test launches the production server once in normal mode and once with the public fused flag, then compares the complete generated token trace. It is intentionally a hardware/model regression rather than a normal CI test because the target and draft artifacts total roughly 98 GB.

[cubic-dev-ai]

All reported issues were addressed across 6 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>