crypto(matryoshka): vendor chacha20 fork over ndarray::simd::U32x16, retire simd_crypto.rs

The AdaWorldAPI ChaCha20 "matryoshka": RustCrypto owns the cipher; ndarray owns
only the U32x16 ARX lane. The standalone from-scratch `src/simd_crypto.rs` (raw
_mm512_*/u32x4_* intrinsics) is RETIRED and superseded by a one-backend fork.

vendor/chacha20/ — fork of RustCrypto chacha20 0.9.1 (name/version kept for
[patch]; own [workspace] table, excluded from the parent workspace). Upstream
cipher verbatim (state schedule, counter, XChaCha, RNG, soft/sse2/avx2/neon).
The ONE delta:

  src/backends/ndarray_simd.rs — the transpose block16 (16 blocks in parallel,
  working vector w holds word w of every block across its 16 lanes, counter word
  carries lane-index 0..=15) expressed over ndarray::simd::U32x16: every
  quarter-round is a pure U32x16 + / ^ / rotate_left, NO cross-lane shuffle, NO
  raw intrinsics, NO unsafe (all of that lives once inside ndarray::simd). Wrapped
  in RustCrypto's StreamBackend (ParBlocksSize = U16), generic over R.

Selected at compile time under cfg(all(target_arch="x86_64", target_feature=
"avx512f")) via four cfg branches in backends.rs + lib.rs (Tokens / new / dispatch).
The ndarray dep is target.'cfg(target_arch="x86_64")' + default-features=false,
features=["std"] so wasm/aarch64 fork builds never pull ndarray.

[patch.crates-io] chacha20 = { path = "vendor/chacha20" } folds it transitively
under chacha20poly1305 -> encryption with zero change to the AEAD.

Parity — triple-gated, GREEN:
  - the fork's own RustCrypto RFC 8439 vectors (chacha20/xchacha20 encryption +
    keystream, core, seek) pass THROUGH ndarray_simd under -Ctarget-cpu=x86-64-v4;
  - cargo test -p encryption (23 AEAD tests incl. XChaCha20-Poly1305 round-trip,
    bit-flip-fails, wrong-key-fails) passes on the default v3 build (RustCrypto
    avx2 fallback) and the avx512 build (ndarray_simd).

Retired: deleted src/simd_crypto.rs + tests/chacha20_rustcrypto_parity.rs + the
chacha20="0.9" dev-dep; removed the ndarray::simd::{chacha20_block,
chacha20_keystream, chacha20_state} surface (zero downstream consumers). aead.rs
doc rewritten to the transitive-acceleration framing.

Note: the workspace default is target-cpu=x86-64-v3 (avx2), so ndarray_simd
activates only on an avx512 build; the wasm-fork backend + cross-repo [patch]
propagation (MedCare-rs) are documented follow-ups in
.claude/CHACHA20_MATRYOSHKA_PLAN.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6pT32kk6pnuAAqR3JiYqu

Author: claude

.claude/CHACHA20_MATRYOSHKA_PLAN.md

@@ -32,10 +32,54 @@ CI: `wasm_simd` job (`.github/workflows/ci.yaml` + `scripts/wasm-parity.sh` +
 lane-by-lane parity selfcheck under **node** — the standing guard for the wasm
 tier (invisible to the x86 `cargo test`). Extend its per-lane blocks as lanes land.
 
-The stand-alone `src/simd_crypto.rs` (`chacha20_block`/`chacha20_keystream`,
-scalar+AVX-512+wasm128, RustCrypto-parity-proven) is the *interim* primitive; the
-matryoshka below **supersedes it** (RustCrypto supplies the rounds, so the
-from-scratch cipher retires once the fork lands).
+The stand-alone `src/simd_crypto.rs` was the *interim* primitive; the matryoshka
+below **superseded it** — it is now **RETIRED** (2026-07-12, see below).
+
+## ✅ SHIPPED 2026-07-12 — fork landed + `simd_crypto.rs` retired
+
+- **Native NEON `U32x16 = [U32x4; 4]`** — `U32x4` gained `bitxor`(veorq_u32) +
+  `rotate_left`(vshlq_u32 shift-or); composed `U32x16` with Add/BitXor/
+  rotate_left. aarch64 now `cargo check`-clean on stable (also fixed the
+  pre-existing `u16x8` alias + the nightly-only `vdotq_s32` breakage → stable
+  widening NEON). Every tier of the ARX lane is now native.
+- **The fork** lives at `vendor/chacha20/` (name/version kept `chacha20`/`0.9.1`,
+  excluded from the workspace, own `[workspace]` table). Upstream cipher verbatim;
+  the ONE delta is `src/backends/ndarray_simd.rs` — the transpose block16 (16
+  blocks ∥, word→16-lane vector, counter lane-index) over `ndarray::simd::U32x16`,
+  pure `+`/`^`/`rotate_left`, **no raw intrinsics, no `unsafe`**. Selected at
+  compile time under `cfg(all(x86_64, avx512f))` in `backends.rs` +
+  `lib.rs::process_with_backend` (Tokens/new/dispatch — four cfg branches). The
+  `ndarray` dep is `target.'cfg(target_arch="x86_64")'` + `default-features=false,
+  features=["std"]` so wasm/aarch64 fork builds never pull ndarray.
+- **`[patch.crates-io] chacha20 = { path = "vendor/chacha20" }`** in the ndarray
+  root `Cargo.toml` folds it under `chacha20poly1305 → encryption` transparently.
+- **Parity — triple-gated, GREEN:** the fork's own RustCrypto RFC 8439 vectors
+  (`chacha20_encryption`/`_keystream`, `xchacha20_*`, `chacha20_core`, seek) pass
+  **through `ndarray_simd`** under `-Ctarget-cpu=x86-64-v4`; `cargo test -p
+  encryption` (23 AEAD tests incl. XChaCha20-Poly1305 round-trip / bit-flip /
+  wrong-key) passes on both the default (v3→RustCrypto avx2 fallback) and avx512
+  (→`ndarray_simd`) builds.
+- **RETIRED:** deleted `src/simd_crypto.rs` + `tests/chacha20_rustcrypto_parity.rs`
+  + the `chacha20 = "0.9"` dev-dep; removed the `ndarray::simd::{chacha20_block,
+  chacha20_keystream, chacha20_state}` surface. RustCrypto owns the cipher; ndarray
+  owns only the `U32x16` lane. `aead.rs` doc updated to the transitive-acceleration
+  framing.
+
+### Reality note + remaining follow-ups
+- **The workspace default is `target-cpu=x86-64-v3` (AVX2), NOT v4** — the CLAUDE.md
+  "v4 mandatory" line is stale. So `ndarray_simd` activates only on an **avx512
+  build** (`.cargo/config-avx512.toml` / `-Ctarget-cpu=x86-64-v4`); the default v3
+  build uses RustCrypto's own avx2 backend (fast, vetted). This is correct and
+  safe — the matryoshka is the *server-avx512* accelerator.
+- **wasm browser backend (next):** add a sibling `#[cfg(all(target_arch="wasm32",
+  target_feature="simd128"))]` branch selecting an `ndarray_simd` backend (same
+  `U32x16` source; the wasm arm is native `[U32x4;4]`) + a `target.'cfg(wasm32)'`
+  ndarray dep. Gated on verifying `ndarray/std` builds inside the encryption
+  `cdylib` for `wasm32-unknown-unknown`.
+- **cross-repo `[patch]` (next):** `[patch]` does not transit, so MedCare-rs (and
+  any other consumer building `encryption` for avx512) needs its own
+  `[patch.crates-io] chacha20 = { path = "vendor/chacha20" }` pointing at a vendored
+  copy of the fork, to inherit the acceleration. Documented, low-risk.
 
 ## DEFERRED — TO-DO (next session)
 

Cargo.lock

@@ -217,11 +217,6 @@ approx = { workspace = true, default-features = true }
 itertools = { workspace = true }
 ndarray-gen = { workspace = true }
 criterion = { version = "0.5", features = ["html_reports"] }
-# The vetted RustCrypto ChaCha20 stream cipher — used ONLY as the independent
-# oracle for the `chacha20_rustcrypto_parity` integration test that pins
-# `ndarray::simd::chacha20_keystream` byte-for-byte against a trusted reference.
-# Same version chacha20poly1305 0.10 pulls transitively (see Cargo.lock).
-chacha20 = "0.9"
 
 [[bench]]
 name = "append"
@@ -407,7 +402,7 @@ members = [
     "ndarray-rand",
     "crates/*",
 ]
-exclude = ["crates/burn", "crates/wasm-simd-parity"]
+exclude = ["crates/burn", "crates/wasm-simd-parity", "vendor/chacha20"]
 default-members = [
     ".",
     "ndarray-rand",
@@ -449,3 +444,15 @@ tag-name = "{{version}}"
 features = ["approx", "serde", "rayon"]
 # Define the configuration attribute `docsrs`
 rustdoc-args = ["--cfg", "docsrs"]
+
+# ── ChaCha20 matryoshka (AdaWorldAPI fork) ──────────────────────────────────
+# Fold the AVX-512/ndarray::simd-accelerated `chacha20` fork over the registry
+# crate. This transitively accelerates the `encryption` crate's XChaCha20-Poly1305
+# keystream (chacha20poly1305 -> chacha20) with zero change to the AEAD, on any
+# x86_64+avx512f build (the workspace's `target-cpu=x86-64-v4`). Non-x86 builds of
+# the fork fall through to RustCrypto's own backends and never pull ndarray.
+# NOTE: a "Patch `chacha20` was not used in the crate graph" note appears on
+# scoped builds that don't pull `encryption` (e.g. `-p ndarray`); it is benign —
+# the patch is exercised by `cargo test -p encryption` and the full workspace.
+[patch.crates-io]
+chacha20 = { path = "vendor/chacha20" }

Cargo.toml

@@ -6,18 +6,18 @@
 //! and a server. Callers normally use [`crate::envelope`], which manages
 //! nonce generation and layout; this module is the raw primitive.
 //!
-//! ## Deliberately NOT hand-composed from `ndarray::simd::chacha20_keystream`
+//! ## Acceleration is transitive (the matryoshka), never hand-composed
 //!
-//! `ndarray` now ships a hardware-accelerated ChaCha20 keystream
-//! (`ndarray::simd::chacha20_keystream`, AVX-512 / wasm128, byte-parity-proven
-//! against RustCrypto). It is tempting to swap this AEAD's keystream to that
-//! primitive for speed — **do not.** XChaCha20-Poly1305 is not just a keystream:
-//! it is HChaCha20 subkey derivation + the Poly1305 one-time-key + MAC framing +
-//! the AAD/length encoding. Re-wiring only the keystream means re-implementing
-//! that authenticated composition by hand, which is exactly the "roll your own
-//! AEAD" footgun the stack forbids. The vetted RustCrypto `XChaCha20Poly1305`
-//! construction stays here. The accelerated primitive is for *raw-stream* use
-//! sites whose caller already owns a vetted MAC/framing — not this module.
+//! This AEAD stays 100% vetted RustCrypto `XChaCha20Poly1305` — HChaCha20 subkey
+//! derivation + Poly1305 one-time-key + MAC framing + AAD/length encoding — and
+//! we do **not** hand-wire a raw keystream into it (that would be the "roll your
+//! own AEAD" footgun the stack forbids). Hardware acceleration instead arrives
+//! *underneath*, transparently: the AdaWorldAPI `chacha20` fork (`vendor/chacha20/`)
+//! replaces one backend of the transitive `chacha20` dependency with a keystream
+//! double-round expressed over `ndarray::simd::U32x16` (the AVX-512 lane), folded
+//! in via `[patch.crates-io]`. So `chacha20poly1305 -> chacha20 -> ndarray::simd`
+//! accelerates on an AVX-512 build with zero change to this module, and every
+//! RustCrypto RFC 8439 test vector (incl. XChaCha20) still passes through it.
 
 use chacha20poly1305::aead::{Aead, KeyInit, Payload};
 use chacha20poly1305::{Key, XChaCha20Poly1305, XNonce};

crates/encryption/src/aead.rs

@@ -246,12 +246,10 @@ pub(crate) mod simd_avx512;
 #[allow(clippy::all, missing_docs, dead_code, unused_variables, unused_imports)]
 pub mod simd_avx2;
 
-// Crypto ARX primitives (ChaCha20 block; Poly1305 / Blake2 to follow) — the
-// hardware-acceleration layer the Ada `encryption` crate draws its hot-path
-// keystream from. Scalar reference + RFC 8439 KAT land first (arch-agnostic,
-// no cfg gate); the AVX-512 / wasm128 / NEON vectorized backends slot in behind
-// the same signature, parity-checked against the KAT (W1a contract).
-pub mod simd_crypto;
+// (The standalone `simd_crypto` ChaCha20 primitive was RETIRED — superseded by
+// the AdaWorldAPI `chacha20` fork whose backend rides `ndarray::simd::U32x16`,
+// `[patch]`ed under the `encryption` AEAD. RustCrypto owns the cipher; ndarray
+// owns only the U32x16 ARX lane. See `vendor/chacha20/` + `.claude/CHACHA20_MATRYOSHKA_PLAN.md`.)
 
 // Portable-SIMD backend — nightly-only. Wraps `core::simd::*` so miri can
 // execute the polyfill paths (intrinsic-based backends are opaque to

src/lib.rs

@@ -682,12 +682,10 @@ pub use crate::simd_ops::{
     sub_f32_inplace,
 };
 
-// ChaCha20 ARX keystream — the crypto hot path the `encryption` AEAD draws from.
-// `chacha20_block` is the scalar reference (RFC 8439 KAT); `chacha20_keystream`
-// is the runtime-dispatched accelerated fill (AVX-512 16-block strides + scalar
-// tail), byte-parity-pinned to the reference. Consumers pull from
-// `ndarray::simd::*` per the W1a contract.
-pub use crate::simd_crypto::{chacha20_block, chacha20_keystream, chacha20_state};
+// ChaCha20 keystream is NO LONGER an `ndarray::simd` surface: the AdaWorldAPI
+// `chacha20` fork (`vendor/chacha20/`) carries the accelerated backend, riding
+// the `U32x16` ARX lane above, and is `[patch]`ed transitively under the
+// `encryption` AEAD. RustCrypto owns the cipher; ndarray exposes only the lane.
 
 // ============================================================================
 // Tests