simd(neon): native U32x16 = [U32x4;4] ARX lane + fix aarch64 stable compile

Native NEON U32x16 (the ChaCha20/BLAKE ARX lane), mirroring the wasm
`[U32x4; 4]` shape so one ChaCha20 backend source compiles unchanged on
every tier:

- U32x4 gains `bitxor` (veorq_u32) + `rotate_left` (shift-or via the
  variable-count vshlq_u32; n%32 early-return). Rotate amount is a public
  ARX constant.
- New U32x16([U32x4;4]) with Add / BitXor / rotate_left / splat /
  from_array / to_array, fanned over the 4 native lanes.
- aarch64_simd F32x16::to_bits/from_bits switch to U32x16::from_array /
  to_array (native is [U32x4;4], not a [u32;16] tuple).
- simd.rs aarch64 arm re-exports the native u32x16/U32x16 from simd_neon
  instead of the scalar fallback (drops them from the scalar list).

Also fixes the pre-existing aarch64 *stable* compile breakage (the target
never built on stable — no aarch64 CI caught it), a prerequisite for the
cross-parity CI job:

- Missing `pub type u16x8 = U16x8;` alias (simd.rs imported a symbol that
  did not exist).
- `dot_i8x16_neon` / `codebook_gather_i8_dotprod` used the nightly-only
  `vdotq_s32` intrinsic (issue #117224). Rewritten with stable widening
  NEON (vmull_s8 + vpaddlq_s16 reduce; vmovl_s8 + vaddw_s16 accumulate) —
  bit-identical result, runs on all aarch64, no `dotprod` feature needed.
  Also fixes a latent bug in the gather (acc1..3 were never written, v1
  aliased v0).

Host ARX parity test green; aarch64 lib now `cargo check`-clean on stable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6pT32kk6pnuAAqR3JiYqu

Author: claude

.claude/blackboard.md

@@ -630,3 +630,27 @@ is superseded by the matryoshka once the fork lands.
 4. **Matryoshka execution:** fork `chacha20`, clone `avx2.rs` backend → rewire over
    `ndarray::simd::U32x16`, `[patch]` into the encryption stack (transitive accel),
    gate vs RustCrypto `soft`, retire `simd_crypto.rs`. Full plan in the doc above.
+
+---
+
+## 2026-07-12 — PR #240 CI fully green (no_std/MSRV fix chain closed)
+
+Tip `5a914c37`. All 20 checks pass; the three previously-red failures are resolved:
+
+- **261f736a** — `simd_crypto` dispatcher: `is_x86_feature_detected!` → compile-time
+  `#[cfg(all(target_arch="x86_64", target_feature="avx512f"))]` (no runtime detection;
+  the workspace SIMD-dispatch rule). Unblocked `blas-msrv` + `nostd/thumbv6m`.
+- **5a914c37** — three no_std/bare-`--no-default-features` test-build fixes:
+  - `simd_crypto` tests: `vec![[0u8;64]; n]` → fixed arrays `[[0u8;64]; 40]` / `[[0u8;64]; 16]`
+    (no `vec!` under no_std).
+  - `tests/chacha20_rustcrypto_parity.rs`: added `#![cfg(feature = "std")]` (imports
+    std-gated `ndarray::simd`; no-ops under `--no-default-features`).
+  - `src/tri.rs`: `Array2::<i32>::zeros(...)` (serde_json dep-drift added
+    `impl PartialEq<Value> for i32`, making bare `Array2::zeros` element-type ambiguous).
+
+Green jobs of note: `tests/{stable,beta,1.95.0}`, `blas-msrv`, `nostd/thumbv6m-none-eabi`,
+`clippy/1.95.0`, `format/stable`, `native-backend/stable`, `tier4-avx512-check`,
+`wasm-simd/parity-node` (new gate), `hpc-stream-parallel/rayon`, CodeRabbit.
+
+Deferred (task #30, token-limit call): native neon `U32x16=[U32x4;4]` + aarch64 cross
+parity CI + the matryoshka chacha20 fork. Plan in `.claude/CHACHA20_MATRYOSHKA_PLAN.md`.

src/simd.rs

@@ -383,10 +383,14 @@ pub use crate::simd_neon::{
 // U16x8 on aarch64 comes from simd_neon (backed by uint16x8_t)
 #[cfg(all(target_arch = "aarch64", not(feature = "nightly-simd")))]
 pub use crate::simd_neon::{u16x8, U16x8};
+// U32x16 (native `[U32x4; 4]`) — the ARX lane the ChaCha20 backend rides. Comes
+// from simd_neon, not the scalar fallback, so it carries Add/BitXor/rotate_left.
+#[cfg(all(target_arch = "aarch64", not(feature = "nightly-simd")))]
+pub use crate::simd_neon::{u32x16, U32x16};
 #[cfg(all(target_arch = "aarch64", not(feature = "nightly-simd")))]
 pub use scalar::{
-    f32x8, f64x4, i32x16, i32x8, i64x4, i64x8, u16x16, u32x16, u32x8, u64x4, u64x8, u8x64, F32x8, F64x4, I32x16, I32x8,
-    I64x4, I64x8, U16x16, U16x32, U32x16, U32x8, U64x4, U64x8, U8x64,
+    f32x8, f64x4, i32x16, i32x8, i64x4, i64x8, u16x16, u32x8, u64x4, u64x8, u8x64, F32x8, F64x4, I32x16, I32x8, I64x4,
+    I64x8, U16x16, U16x32, U32x8, U64x4, U64x8, U8x64,
 };
 
 // wasm32 + simd128: the native v128 float hot path (F32x16 / F64x8 + masks)

src/simd_neon.rs

@@ -183,49 +183,62 @@ pub unsafe fn codebook_gather_f32x4_a72(centroids: &[f32], indices: &[u8], dim:
 // Tier 3: A76 DotProd + FP16 (Pi 5, Orange Pi 5)
 // ═══════════════════════════════════════════════════════════════════════════
 
-/// SDOT: 4×(4×i8 · 4×i8) → 4×i32 in ONE instruction.
-/// ARMv8.2+ dotprod. 4× throughput vs manual widening multiply.
-/// Core of int8 quantized codebook inference on Pi 5.
+/// int8 dot product of two 16-lane chunks → i32.
+///
+/// Stable-Rust widening path: `vmull_s8` (8×(i8·i8)→i16 per half) then
+/// `vpaddlq_s16` (pairwise widen i16→i32) and a horizontal `vaddvq_s32`.
+/// Bit-identical result to the ARMv8.2 `SDOT` instruction, but compiles on
+/// stable (the `vdotq_s32` intrinsic is nightly-only, issue #117224) and runs
+/// on **all** aarch64 — no `dotprod` feature required. Max |product| = 16384,
+/// pairwise sums stay well inside i32, so no overflow for any i8 input.
 #[cfg(target_arch = "aarch64")]
-#[target_feature(enable = "dotprod")]
+#[inline(always)]
 pub unsafe fn dot_i8x16_neon(a: &[i8; 16], b: &[i8; 16]) -> i32 {
     let va = vld1q_s8(a.as_ptr());
     let vb = vld1q_s8(b.as_ptr());
-    let acc = vdupq_n_s32(0);
-    let result = vdotq_s32(acc, va, vb);
-    // Horizontal sum of 4×i32
-    vaddvq_s32(result)
+    // 8×i16 products for the low and high halves.
+    let plo = vmull_s8(vget_low_s8(va), vget_low_s8(vb));
+    let phi = vmull_s8(vget_high_s8(va), vget_high_s8(vb));
+    // Widen i16→i32 (pairwise) so the accumulation never overflows, then reduce.
+    vaddvq_s32(vaddq_s32(vpaddlq_s16(plo), vpaddlq_s16(phi)))
 }
 
-/// Quantized codebook gather via SDOT (Pi 5 only).
-/// Centroids stored as i8, accumulated as i32. 4× throughput vs f32 path.
+/// Quantized codebook gather: element-wise widen-accumulate the selected i8
+/// centroids into an i32 output of length `dim` (`dim` a multiple of 16).
+///
+/// The i32 counterpart of [`codebook_gather_f32x4_neon`]: for each output lane
+/// `k`, `output_i32[k] = Σ_idx centroids_i8[idx*dim + k]`, widened i8→i32 via
+/// `vmovl_s8` + `vaddw_s16`. Stable on all aarch64 (no `dotprod` intrinsic).
 #[cfg(target_arch = "aarch64")]
-#[target_feature(enable = "dotprod")]
+#[inline(always)]
 pub unsafe fn codebook_gather_i8_dotprod(
     centroids_i8: &[i8], // quantized centroids: N × dim (i8)
     indices: &[u8],
     dim: usize,             // must be multiple of 16
     output_i32: &mut [i32], // accumulated i32 (dequantize later)
 ) {
     debug_assert!(dim % 16 == 0);
+    debug_assert!(output_i32.len() >= dim);
     let chunks = dim / 16;
 
     for c in 0..chunks {
+        // Four i32x4 accumulators cover the 16 i8 lanes of this chunk.
         let mut acc0 = vdupq_n_s32(0);
         let mut acc1 = vdupq_n_s32(0);
         let mut acc2 = vdupq_n_s32(0);
         let mut acc3 = vdupq_n_s32(0);
 
         for &idx in indices {
             let base = idx as usize * dim + c * 16;
-            let v0 = vld1q_s8(centroids_i8[base..].as_ptr());
-            let v1 = vld1q_s8(centroids_i8[base..].as_ptr());
-            // dotprod: each vdotq_s32 does 4×(4×i8·4×i8)→4×i32
-            let ones = vdupq_n_s8(1); // identity for accumulation
-            acc0 = vdotq_s32(acc0, v0, ones);
+            let v = vld1q_s8(centroids_i8[base..].as_ptr()); // 16 i8
+            let lo = vmovl_s8(vget_low_s8(v)); // lanes 0..8  → i16
+            let hi = vmovl_s8(vget_high_s8(v)); // lanes 8..16 → i16
+            acc0 = vaddw_s16(acc0, vget_low_s16(lo));
+            acc1 = vaddw_s16(acc1, vget_high_s16(lo));
+            acc2 = vaddw_s16(acc2, vget_low_s16(hi));
+            acc3 = vaddw_s16(acc3, vget_high_s16(hi));
         }
 
-        // Store 4 i32 results
         vst1q_s32(output_i32[c * 16..].as_mut_ptr(), acc0);
         vst1q_s32(output_i32[c * 16 + 4..].as_mut_ptr(), acc1);
         vst1q_s32(output_i32[c * 16 + 8..].as_mut_ptr(), acc2);
@@ -467,7 +480,11 @@ pub mod aarch64_simd {
 
     // Integer types come from the scalar fallback in simd.rs — they aren't on
     // the perf-critical f32 BLAS-1 / VML path that this module accelerates.
-    pub use crate::simd::scalar::{I32x16, U32x16, U64x8};
+    // `U32x16` is the exception: it carries the ARX vocabulary (Add/BitXor/
+    // rotate_left) the ChaCha20 lane needs, so it is the native `[U32x4; 4]`
+    // defined at the top of this file (mirroring `simd_wasm::wasm32_simd`).
+    pub use super::U32x16;
+    pub use crate::simd::scalar::{I32x16, U64x8};
 
     /// 16×f32 backed by 4× NEON `float32x4_t` registers (paired loads).
     #[derive(Copy, Clone)]
@@ -674,13 +691,14 @@ pub mod aarch64_simd {
             for i in 0..16 {
                 o[i] = a[i].to_bits();
             }
-            U32x16(o)
+            U32x16::from_array(o)
         }
         #[inline(always)]
         pub fn from_bits(bits: U32x16) -> Self {
+            let b = bits.to_array();
             let mut o = [0.0f32; 16];
             for i in 0..16 {
-                o[i] = f32::from_bits(bits.0[i]);
+                o[i] = f32::from_bits(b[i]);
             }
             Self::from_array(o)
         }
@@ -1490,6 +1508,11 @@ impl U16x8 {
     }
 }
 
+// Lowercase alias (consumer-API parity — `simd.rs` re-exports `u16x8`).
+#[cfg(target_arch = "aarch64")]
+#[allow(non_camel_case_types)]
+pub type u16x8 = U16x8;
+
 #[cfg(target_arch = "aarch64")]
 #[derive(Copy, Clone)]
 #[repr(transparent)]
@@ -1542,8 +1565,122 @@ impl U32x4 {
     pub fn max(self, other: Self) -> Self {
         Self(unsafe { vmaxq_u32(self.0, other.0) })
     }
+    /// Lane-wise XOR — the ARX `⊕` (`veorq_u32`).
+    #[inline(always)]
+    pub fn bitxor(self, other: Self) -> Self {
+        Self(unsafe { veorq_u32(self.0, other.0) })
+    }
+    /// Lane-wise left-rotate by `n` bits — the ARX rotate (matches
+    /// `u32::rotate_left`). NEON has no rotate op, so this is the shift-or via
+    /// the variable-count `vshlq_u32` (a signed per-lane count: `+n` shifts
+    /// left by `n`, `n-32 < 0` shifts logical-right by `32-n`). The `n % 32 == 0`
+    /// early-return avoids the ambiguous full-width shift. Rotate amount is a
+    /// public ARX constant.
+    #[inline(always)]
+    pub fn rotate_left(self, n: u32) -> Self {
+        let n = n % 32;
+        if n == 0 {
+            return self;
+        }
+        unsafe {
+            let l = vshlq_u32(self.0, vdupq_n_s32(n as i32));
+            let r = vshlq_u32(self.0, vdupq_n_s32(n as i32 - 32));
+            Self(vorrq_u32(l, r))
+        }
+    }
 }
 
+/// 16×u32 as `[U32x4; 4]` — the NEON-native 16-wide ARX lane (ChaCha20 /
+/// BLAKE). `U32x4` is the dispatched native unit (`uint32x4_t`); `U32x16` fans
+/// each op over the 4 sub-lanes. Consumer API (`Add` / `BitXor` / `rotate_left`)
+/// matches `simd_avx512::U32x16` exactly, so the ChaCha20 backend compiles
+/// unchanged on every tier (the wasm arm uses the identical `[U32x4; 4]` shape).
+#[cfg(target_arch = "aarch64")]
+#[derive(Copy, Clone)]
+#[repr(align(64))]
+pub struct U32x16(pub [U32x4; 4]);
+
+#[cfg(target_arch = "aarch64")]
+impl U32x16 {
+    pub const LANES: usize = 16;
+
+    #[inline(always)]
+    pub fn splat(v: u32) -> Self {
+        Self([U32x4::splat(v); 4])
+    }
+
+    #[inline(always)]
+    pub fn from_array(a: [u32; 16]) -> Self {
+        Self([
+            U32x4::from_array([a[0], a[1], a[2], a[3]]),
+            U32x4::from_array([a[4], a[5], a[6], a[7]]),
+            U32x4::from_array([a[8], a[9], a[10], a[11]]),
+            U32x4::from_array([a[12], a[13], a[14], a[15]]),
+        ])
+    }
+
+    #[inline(always)]
+    pub fn to_array(self) -> [u32; 16] {
+        let mut o = [0u32; 16];
+        for i in 0..4 {
+            o[i * 4..i * 4 + 4].copy_from_slice(&self.0[i].to_array());
+        }
+        o
+    }
+
+    /// Lane-wise left-rotate by `n` bits (ARX rotate), fanned over 4 lanes.
+    #[inline(always)]
+    pub fn rotate_left(self, n: u32) -> Self {
+        Self([
+            self.0[0].rotate_left(n),
+            self.0[1].rotate_left(n),
+            self.0[2].rotate_left(n),
+            self.0[3].rotate_left(n),
+        ])
+    }
+}
+
+#[cfg(target_arch = "aarch64")]
+impl core::ops::Add for U32x16 {
+    type Output = Self;
+    #[inline(always)]
+    fn add(self, r: Self) -> Self {
+        Self([self.0[0].add(r.0[0]), self.0[1].add(r.0[1]), self.0[2].add(r.0[2]), self.0[3].add(r.0[3])])
+    }
+}
+
+#[cfg(target_arch = "aarch64")]
+impl core::ops::BitXor for U32x16 {
+    type Output = Self;
+    #[inline(always)]
+    fn bitxor(self, r: Self) -> Self {
+        Self([
+            self.0[0].bitxor(r.0[0]),
+            self.0[1].bitxor(r.0[1]),
+            self.0[2].bitxor(r.0[2]),
+            self.0[3].bitxor(r.0[3]),
+        ])
+    }
+}
+
+#[cfg(target_arch = "aarch64")]
+impl core::fmt::Debug for U32x16 {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(f, "U32x16({:?})", self.to_array())
+    }
+}
+
+#[cfg(target_arch = "aarch64")]
+impl PartialEq for U32x16 {
+    fn eq(&self, other: &Self) -> bool {
+        self.to_array() == other.to_array()
+    }
+}
+
+#[cfg(target_arch = "aarch64")]
+#[allow(non_camel_case_types)]
+pub type u32x16 = U32x16;
+
 #[cfg(target_arch = "aarch64")]
 #[derive(Copy, Clone)]
 #[repr(transparent)]