Misc. bug: llama-quantize fails on DeepSeek-V4 — i32 ffn_gate_tid2eid routing table not excluded from quantization

[KikoCis]

Summary

llama-quantize fails on DeepSeek-V4 models because the arch's ffn_gate_tid2eid tensor is i32 and isn't excluded from quantization.

Repro

Convert any DeepSeek-V4 checkpoint (e.g. deepseek-ai/DeepSeek-V4-Flash-DSpark) and quantize it:

python convert_hf_to_gguf.py DeepSeek-V4-Flash-DSpark --outfile v4-q8.gguf --outtype q8_0 --fp8-as-q8
llama-quantize --allow-requantize v4-q8.gguf v4-q4.gguf Q4_K_M

Output:

[  20/1328] blk.0.ffn_gate_shexp.weight  - [4096, 2048, 1, 1], type = q8_0, converting to q4_K .. size = 8.50 MiB -> 4.50 MiB
[  21/1328] blk.0.ffn_gate_tid2eid.weight - [   6, 129280, 1, 1], type = i32,
llama_model_quantize: failed to quantize: cannot dequantize/convert tensor type i32

(thrown at src/llama-quant.cpp:228)

Cause

blk.N.ffn_gate_tid2eid.weight (LLM_TENSOR_FFN_GATE_TID2EID, declared in src/llama-arch.cpp and created in src/models/deepseek4.cpp as {n_expert_used, n_vocab}) is a token-id → expert-id index table, not weights. It's i32, so llama_tensor_quantize_impl can't convert it.

llama_tensor_get_type() in src/llama-quant.cpp already excludes the other MoE routing tensor:

quantize &= name.find("ffn_gate_inp.weight") == std::string::npos;

but tid2eid was never added when DeepSeek-V4 landed (#24162) — understandable, since that work targeted inference.

Suggested fix

One line, next to the existing router exclusion:

quantize &= name.find("ffn_gate_inp.weight") == std::string::npos;
quantize &= name.find("tid2eid") == std::string::npos;   // DeepSeek-V4: i32 token-id -> expert-id table

With this, the tensor is copied through verbatim (2.959 MiB) and quantization completes.

I'm happy to open a PR if that's useful — just say the word (or feel free to take the line directly, it's trivial).

Notes

• Probably went unnoticed because the public V4 releases already ship quantized (FP8 attention + MXFP4 experts, ~4.4 BPW), so there's little reason to run llama-quantize on them. It does matter if you want to push the routed experts below 4 bits.
• Tested on 505b1ed1, macOS/Metal.

Thanks for the V4 support — it landed fast and the conversion path (incl. --fp8-as-q8) worked flawlessly otherwise.