feat: add tract_moe_ffn operator for Mixture-of-Experts FFN

[JulienBalianSonos]

Implements the tract_moe_ffn operator in the tract_transformers extension, enabling inference of routed Mixture-of-Experts FFN blocks exported via torch_to_nnef.

The operator encapsulates the full MoE FFN block:

• Router: x @ wg.T followed by top-k expert selection.
• Gating modes:
  • softmax over the selected top-k experts
  • softmax over all experts, then gather top-k without renormalization
  • raw / already-normalized top-k weights
• Token grouping: batch selected tokens per expert for efficient GEMM.
• Expert FFN: activation(x @ w1) * (x @ w3) followed by @ w2 for SwiGLU-style experts.
• Weighted scatter-add of expert outputs.

Real conditional compute: unused experts are fully skipped. The op handles both 2D [T,D] and 3D [B,S,D] input shapes.

Model Coverage

The implemented semantics cover the routed FFN variants used by:

• Granite / Mixtral-style MoE: top-k routing, softmax-over-top-k gating, SwiGLU experts.
• OLMoE-style MoE: router softmax over all experts, gather top-k, no top-k renormalization.
• OpenAI GPT-OSS-style MoE: optional router/expert biases and clamped SwiGLU.
• Qwen3 / Qwen3.5-style MoE: top-k routing with configurable top-k probability normalization.

Current validation:

• Bit-exact against PyTorch on TitanML/tiny-mixtral (8 experts, top-2, 246M params).
• End-to-end NNEF regression on a sanitized Qwen3 tiny MoE graph.
• Full-model local validation on a Granite MoE GPTQ/Q40 export with Q40 experts and embeddings, keeping routing in full precision.

Granite MoE Benchmark

Local release measurements on Apple Silicon. The benchmark artifact is not included in this PR.

Model / path	Backend	Prompt / decode	KV path	Throughput
Granite MoE GPTQ/Q40 experts + embeddings	Metal	5 / 64	unfolded	77.68 tok/s
Granite MoE GPTQ/Q40 experts + embeddings	Metal	5 / 64	stateful	78.66 tok/s
Granite MoE GPTQ/Q40 experts + embeddings	Metal	51 / 128	unfolded	73.86 tok/s
Granite MoE GPTQ/Q40 experts + embeddings	Metal	51 / 128	stateful	74.47 tok/s
Granite MoE GPTQ/Q40 experts + embeddings	Metal	51 / 512	unfolded	65.04 tok/s
Granite MoE GPTQ/Q40 experts + embeddings	Metal	51 / 512	stateful	65.35 tok/s
Granite MoE GPTQ/Q40 experts + embeddings	CPU	51 / 128	unfolded	40.63 tok/s

Additional steady-state Metal harness results on the same model family:

Variant	PP512	TG128
Q40 MoE no-SDPA reference	1280.3 tok/s	123.5 tok/s
mixed SDPA, f16 input / f32 accumulation	1254.6 tok/s	122.0 tok/s
old SDPA f32-boundary casts	1128.2 tok/s	119.4 tok/s

[github-actions]

🔴 Bench vs main — 1 speed regression(s)

Reference: main nightly, latest 2026-07-09 (0d old) · PR dfd2901db · ran on apple-m1-max, i9-11900kb_rtx-4060, jetson-orin-nx · 791 metrics compared

Speed — evaltime · prefill · decode

Δ	metric	device	main → PR
🔴 -4.8%	openelm_270M_q40ef16_541 decode · cpu	apple-m1-max	79.54 tok/s → 75.75 tok/s

lower is better except prefill/decode (tok/s) · adaptive thresholds (max(floor, k×noise) vs the series' own history) · single-shot vs nightly reference · full report → run