Skip to content

Commit 8d85637

Browse files
committed
simd: verify-pass fixes — backend engine on the unfused reference tier
Three-angle adversarial verify (numerics/swap-trace/docs) on the completion diff. Substantive finding acted on: the fused tier's scalar polyfill (AVX2 arm F64x8::mul_add = per-lane f64::mul_add) lowers to a libm fma() call on baseline x86-64 builds without the fma target feature — downstream consumers do not inherit this repo's .cargo pin, so backend::native::gemm_f64 now routes through the UNFUSED reference tier instead: no libm dependence on any backend, ~7% slower than fused on pinned builds (10.0 vs 10.7 GFLOP/s), and the backend engine is now bit-identical to the certification reference. gemm_f64_tiled_fma stays public for FMA-pinned consumers. Doc fixes from the same pass: removed two stale 'gemm_f64 delegates to matrixmultiply' claims that contradicted the swap (simd.rs re-export comment, gemm_f64_tiled rustdoc); added # Panics to gemm_f64 (checked preconditions vs the old wrapper's silent UB on short slices, matching CBLAS xerbla); scoped fma determinism to per-(build,runtime) — wasm relaxed-simd fusion is implementation-defined; corrected the AVX2 vfmadd naming (per-lane polyfill, fused semantics); fixed the integer-corpus bound comment (k_max=128). Gates: clippy -D warnings clean, 2185/2185 lib tests, 4 gemm doctests.
1 parent ec0373f commit 8d85637

4 files changed

Lines changed: 77 additions & 30 deletions

File tree

.claude/blackboard.md

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,22 @@ below's foundation:
2424
f64→f64 store/reload never rounds ⇒ per-element op sequence
2525
unchanged ⇒ every bit-equality test green untouched. [MEASURED]
2626
ref 4.6→10.0 GF, fma 4.7→10.7 GF (2.2×).
27-
3. **Engine swap:** `backend::native::gemm_f64` now routes to
28-
`gemm_f64_tiled_fma` — the f64 GEMM behind `BlasFloat::backend_gemm`
29-
/ `hpc::blas_level3::blas_gemm` / batched linalg is entirely own
30-
Rust; matrixmultiply remains only in gemm_f32 and upstream
31-
`Array::dot` (impl_linalg.rs, untouched at ~33 GF).
27+
3. **Engine swap:** `backend::native::gemm_f64` now routes to the
28+
crate-native tiled kernel — the f64 GEMM behind
29+
`BlasFloat::backend_gemm` / `hpc::blas_level3::blas_gemm` / batched
30+
linalg is entirely own Rust; matrixmultiply remains only in gemm_f32
31+
and upstream `Array::dot` (impl_linalg.rs, untouched at ~33 GF).
32+
**[REVISED post-verify] Engine = the UNFUSED reference tier**, not
33+
fma: the verify pass surfaced a cliff — the AVX2-polyfill/scalar
34+
`mul_add` lowers to a libm `fma()` call on baseline x86-64 builds
35+
(consumers do NOT inherit this repo's `.cargo` target-cpu pin; CI
36+
lands exactly there). The unfused tier has no libm dependence on any
37+
backend, costs only ~7% vs fused on pinned builds (10.0 vs 10.7 GF),
38+
and makes the backend engine bit-identical to the certification
39+
reference. `gemm_f64_tiled_fma` stays public for FMA-pinned
40+
consumers. New panic contract documented on `gemm_f64` (# Panics —
41+
checked preconditions vs the old wrapper's silent-UB on short
42+
slices; matches CBLAS xerbla behavior).
3243

3344
[MEASURED, 3-engine, this VM (v3 compile → AVX2 arm, PREFERRED_F64_LANES=4;
3445
host runtime has avx512f but committed .cargo config is v3)]:
@@ -46,6 +57,18 @@ still dead in native.rs `mod scalar` (f32 sibling completion);
4657
avx512f compile arm untested on CI (v3 config) — the F64x8=__m512d arm
4758
runs only on local v4 builds.
4859

60+
**[VERIFY OUTCOME]** 3-angle adversarial pass on the completion diff:
61+
numerics PASS / swap-trace PASS / docs FAIL→fixed. Substantive P1 acted
62+
on: baseline-x86 libm-fma cliff → engine revised to the unfused
63+
reference tier (see #3 REVISED above). Doc P1s fixed: two stale
64+
"gemm_f64 delegates to matrixmultiply" claims (simd.rs comment +
65+
gemm_f64_tiled rustdoc) contradicted the swap in the same diff. P2s
66+
fixed: fma determinism scoped to per-(build,runtime) (wasm relaxed-simd
67+
fusion is implementation-defined); AVX2 vfmadd naming corrected (per-
68+
lane f64::mul_add polyfill, fused semantics); integer-corpus bound
69+
comment corrected (k_max=128, ≈2.1e4). All gates re-run green after
70+
fixes: clippy -D warnings, 2185/2185 lib tests, 4 gemm doctests.
71+
4972
## 2026-07-06 (later) — `ndarray::simd::gemm_f64_tiled` surfaced (operator directive)
5073

5174
The crate-native tiled f64 GEMM graduated from dead code

src/backend/native.rs

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -238,17 +238,30 @@ pub fn gemm_f32(
238238

239239
/// GEMM: C = alpha * A * B + beta * C (f64, row-major).
240240
///
241-
/// Engine: the crate-native `simd_ops::gemm_f64_tiled_fma` (F64x8-vectorized
242-
/// fused tier — entirely in-crate Rust, no external matmul engine). The
243-
/// unfused sibling `ndarray::simd::gemm_f64_tiled` is the bit-exact
244-
/// certification reference; this entry favors throughput. Swapped from
245-
/// `matrixmultiply::dgemm` — results agree to BLAS tolerance (last-ulp
246-
/// accumulation-order/fusion differences), not bit-for-bit.
241+
/// Engine: the crate-native `simd_ops::gemm_f64_tiled` (F64x8-vectorized,
242+
/// unfused reference tier — entirely in-crate Rust, no external matmul
243+
/// engine for f64; `gemm_f32` below still delegates to `matrixmultiply`).
244+
/// The unfused tier is chosen deliberately: it is bit-identical to the
245+
/// certification reference on every backend AND has no dependence on the
246+
/// `fma` target feature (the fused tier's scalar polyfill can lower to a
247+
/// libm `fma()` call on baseline x86-64 builds that don't inherit this
248+
/// repo's `.cargo` target-cpu pin). Consumers that pin an FMA-capable
249+
/// target and want the fused tier call `ndarray::simd::gemm_f64_tiled_fma`
250+
/// directly. Swapped from `matrixmultiply::dgemm` — results agree to BLAS
251+
/// tolerance (last-ulp accumulation-order differences), not bit-for-bit.
252+
///
253+
/// # Panics
254+
///
255+
/// Unlike the previous `matrixmultiply` wrapper (which accepted `lda < k`
256+
/// silently and was UB on short slices), this entry inherits the tiled
257+
/// kernel's checked preconditions: panics if `lda < k`, `ldb < n`,
258+
/// `ldc < n`, or a slice is shorter than `(rows − 1)·ld + cols` — matching
259+
/// the CBLAS backends, which reject `lda < max(1, k)` via `xerbla`.
247260
pub fn gemm_f64(
248261
m: usize, n: usize, k: usize, alpha: f64, a: &[f64], lda: usize, b: &[f64], ldb: usize, beta: f64, c: &mut [f64],
249262
ldc: usize,
250263
) {
251-
crate::simd_ops::gemm_f64_tiled_fma(m, n, k, alpha, a, lda, b, ldb, beta, c, ldc);
264+
crate::simd_ops::gemm_f64_tiled(m, n, k, alpha, a, lda, b, ldb, beta, c, ldc);
252265
}
253266

254267
// ─── GEMV dispatch ───────────────────────────────────────────────

src/simd.rs

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -570,13 +570,15 @@ pub use crate::simd_ops::{array_chunks, array_chunks_checked, array_windows, arr
570570
// contract: unfused mul+add in ascending-k order per element → bit-identical
571571
// on every backend (AVX-512/AVX2/NEON/WASM/scalar) and, at α=1 β=0,
572572
// bit-identical to the naive triple-loop reference. This is the in-crate
573-
// ground-truth GEMM for probes/certification — `backend::native::gemm_f64`
574-
// delegates to the external `matrixmultiply` crate instead. The kernel is
575-
// alloc-free, but `pub mod simd`/`simd_ops` are std-gated in lib.rs, so
576-
// like every kernel here it is reachable only in `std` builds today.
573+
// ground-truth GEMM for probes/certification AND the engine behind
574+
// `backend::native::gemm_f64` (own Rust in the f64 BLAS path; the f32
575+
// sibling still delegates to the external `matrixmultiply` crate).
577576
// `gemm_f64_tiled_fma` is the fast fused tier (same tiling/order, one
578-
// rounding per step) — the engine behind `backend::native::gemm_f64`;
579-
// the unfused reference tier stays the certification ground truth.
577+
// rounding per step) for consumers on FMA-pinned targets — not the
578+
// backend engine, because its scalar polyfill can lower to libm `fma()`
579+
// on baseline builds. Both kernels are alloc-free, but `pub mod
580+
// simd`/`simd_ops` are std-gated in lib.rs, so they are reachable only
581+
// in `std` builds today.
580582
pub use crate::simd_ops::{gemm_f64_tiled, gemm_f64_tiled_fma};
581583
pub use crate::simd_soa::MultiLaneColumn;
582584

src/simd_ops.rs

Lines changed: 21 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -890,10 +890,11 @@ mod add_mul_tests {
890890
/// The crate-native f64 GEMM: a TILE=64-blocked loop nest with the
891891
/// innermost j-run vectorized on the dispatched [`F64x8`] (AVX-512 /
892892
/// AVX2 / NEON / WASM-SIMD128 / scalar — one source, every backend).
893-
/// Unlike `backend::native::gemm_f64`, which delegates to the external
894-
/// `matrixmultiply` crate, this kernel is entirely in-crate and carries
895-
/// a bit-exactness contract, so probes can certify other kernels
896-
/// against it as ground truth.
893+
/// Entirely in-crate with a bit-exactness contract, so probes can
894+
/// certify other kernels against it as ground truth — and since it IS
895+
/// the engine behind `backend::native::gemm_f64`, the native f64 BLAS
896+
/// path carries the same auditable numerics (the f32 sibling still
897+
/// delegates to the external `matrixmultiply` crate).
897898
///
898899
/// # Bit-exactness contract
899900
///
@@ -953,14 +954,21 @@ pub fn gemm_f64_tiled(
953954
/// Fast tier of [`gemm_f64_tiled`]: same tiling, same ascending-`p`
954955
/// per-element order, but each step is a **fused** multiply-add
955956
/// (`c = fma(alpha·A[i,p], B[p,j], c)` — one rounding per step via
956-
/// `F64x8::mul_add`: `vfmadd`/`vfmaq_f64` on x86/NEON, `f64::mul_add`
957-
/// on the scalar tail and polyfill backends).
957+
/// `F64x8::mul_add`; fused semantics on every backend, though the
958+
/// lowering varies: native `vfmadd` on the AVX-512 arm, `vfmaq_f64` on
959+
/// NEON, per-lane `f64::mul_add` on the AVX2 polyfill and the scalar
960+
/// tail — which can become a libm `fma()` call on targets built
961+
/// without the `fma` feature: correct, but slow. Prefer the reference
962+
/// tier, or an FMA-capable target pin, in that situation).
958963
///
959964
/// Semantics relative to the reference tier:
960-
/// - **Deterministic per build**, but NOT bit-stable across backends:
961-
/// fusion quality differs (WASM without `relaxed-simd` has no fused
962-
/// vector `mul_add`, so its vector lanes match the unfused reference
963-
/// while the scalar tail is still fused).
965+
/// - **Deterministic per (build, runtime)**, but NOT bit-stable across
966+
/// backends: fusion differs (WASM without `relaxed-simd` has no
967+
/// fused vector `mul_add`, so its vector lanes match the unfused
968+
/// reference while the scalar tail is still fused; WASM WITH
969+
/// `relaxed-simd` uses `f64x2_relaxed_madd`, whose fusion is
970+
/// implementation-defined — the same wasm binary may round
971+
/// differently on different runtimes).
964972
/// - **Bit-identical to [`gemm_f64_tiled`]** whenever every
965973
/// `a_val·b` product and running sum is exactly representable —
966974
/// e.g. integer-valued data with products and accumulation inside
@@ -1316,8 +1324,9 @@ mod gemm_f64_tiled_tests {
13161324
// ── FMA tier ──
13171325

13181326
/// Integer-valued corpus: values in [-9, 9], so every a·b product
1319-
/// (≤ 81) and running sum (≤ 70·81·2) is exactly representable —
1320-
/// fused and unfused steps round identically → tiers bit-identical.
1327+
/// (≤ 81) and running sum (≤ k_max·81·|α| + 9 = 128·81·2 + 9 ≈ 2.1e4,
1328+
/// far below 2^53) is exactly representable — fused and unfused
1329+
/// steps round identically → tiers bit-identical.
13211330
fn fill_int(len: usize, salt: u64) -> Vec<f64> {
13221331
(0..len)
13231332
.map(|i| ((mix(i as u64 ^ salt) % 19) as i64 - 9) as f64)

0 commit comments

Comments
 (0)