block_quant: role-aware weight quantization policy

[czoli1976]

What this does

Gates the block_quant transform on a role-aware policy instead of quantizing every constant matmul weight uniformly: quant-sensitive tensors (embeddings, normalizations, output heads) and matmuls too small to benefit are left in f32, and the rest are quantized to Q4_0.

fn should_block_quant(weight_name: &str, k: usize, n: usize) -> bool {
    const PROTECTED: &[&str] =
        &["embed", "Embed", "norm", "Norm", "pooler", "classifier", "lm_head", "logits"];
    !PROTECTED.iter().any(|p| weight_name.contains(p)) && k * n >= 1 << 14
}

Why

Quantizing every weight uniformly degrades quality on the few sensitive tensors (embeddings/norms/heads are small and numerically sensitive — quantizing them costs accuracy and saves almost nothing) while small matmuls aren't worth the overhead. Protecting them by name + a size floor is the convention llama.cpp / GPTQ / bitsandbytes all converge on. The weight names survive ONNX import, so the policy can key on them.

Tests: protected/tiny weights are skipped, the rest are quantized (L1), and the quantized result matches X @ Q4_0(W) across both ONNX orientations (L2).

Notes

• Stacked on block_quant: quantize einsum weights at either input slot #2428 — the first commit is that block_quant slot-orientation fix (its dependency); the policy is the second commit.
• This is the "which-tensors" axis only. The asymmetric bit-width half of the recipe (bump attn_v/ffn_down to 8-bit) is a natural follow-up once a Q8_1-weight matmul path exists.

Question for you

This bakes a name-based protected-list into core. Is that the shape you'd want, or would you rather it be configurable (transform params / a caller-supplied predicate or regex list), or kept out of core entirely? Happy to refactor toward whatever you prefer — opening as a draft to get your call before polishing.

🍍