Pre-release closeout: crate-wide unsafe lint, CI fuzz smoke, rank_to_bucket fail-loud#72
Conversation
Move #![deny(unsafe_op_in_unsafe_fn)] from fastscan.rs to the crate root
(lib.rs) so every unsafe operation in the SIMD kernels must sit in an
explicit `unsafe {}` block rather than leaning on an enclosing `unsafe fn`.
Wraps the AVX-512/AVX2 kernel bodies in bitmap.rs, sign_bitmap.rs and
quant_kernels.rs, and the NEON popcount in util.rs. horizontal_sum_avx2 is
register-only (no memory access), so its intrinsics are safe under the
#[target_feature] gate and need no block (an explicit one would be
unused_unsafe). Purely additive — no kernel logic changes. Keeps the unsafe
surface visible to future edits (THREAT-SIMD-001).
Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
Adds .github/workflows/fuzz.yml: a 60s/target smoke over load_rank, load_rankquant and fastscan_b2 on every pull request / push to main, plus a weekly full sweep over all seven targets. Surfaces loader / write->load round-trip / FastScan-kernel regressions in CI between manual campaigns (THREAT-FUZZ-002). cargo-fuzz is version-pinned (0.13.1) and every action is SHA-pinned; read-only token; the matrix target is passed via env (no run: injection surface, per THREAT-CICD-001). Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
rank_to_bucket clamped the computed bucket with `b.min(n_buckets - 1)`, silently accepting rank >= d. A valid rank is a position in [0, d), so it now asserts `rank < d` and drops the clamp — matching the fail-loud contract of the sibling primitives pack_buckets / bucket_centre (#26). Valid rank vectors (a permutation of [0, d)) are unaffected. The Python bindings gain matching guards on rank_to_bucket and bucket_ranks so the new assert surfaces as a clean ValueError rather than a PanicException, with red-team tests for both. Also refreshes the stale pack_buckets binding comment (the core now asserts in-range, no longer masks). Closes #28. Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
Threat model + RELEASING now reflect the in-place controls: - SUPPLY-002: GitHub immutable releases enabled + main branch protection (PR review required, force-pushes and deletions blocked), closing the release-tag/asset mutability surface the registries already close. - SIMD-001: unsafe_op_in_unsafe_fn now denied crate-wide. - FUZZ-002: fuzz.yml CI smoke + weekly sweep. Corrects the workflow count (12 -> 13) and adds the CHANGELOG entries. Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
cargo-fuzz 0.13.1's bundled Cargo.lock pins rustix 0.36.x, which no longer compiles on current nightly (rustc_layout_scalar_valid_range_start is now a reserved attribute), so 'cargo install --locked' failed the three fuzz smoke jobs. Drop --locked; the tool stays version-pinned (0.13.1) and its build deps resolve to nightly-compatible versions. Verified locally on nightly. Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
There was a problem hiding this comment.
Code Review
This pull request rolls out the #![deny(unsafe_op_in_unsafe_fn)] lint crate-wide, wrapping all unsafe operations in explicit unsafe {} blocks across the SIMD kernels (bitmap.rs, sign_bitmap.rs, quant_kernels.rs, and util.rs). It also updates rank_to_bucket to fail loud and panic when rank >= d instead of silently clamping, with corresponding clean ValueError handling added to the Python bindings. Additionally, the threat model, changelog, and releasing documentation have been updated to reflect these mitigations and the newly added CI fuzzing workflows. I have no feedback to provide as there are no review comments.
|
/agentic_review |
There was a problem hiding this comment.
Pull request overview
This PR closes pre-release hardening items by expanding unsafe-operation linting crate-wide, adding bounded fuzzing CI, and tightening rank_to_bucket/binding behavior to fail loudly on invalid ranks.
Changes:
- Moves
unsafe_op_in_unsafe_fnenforcement to the crate root and wraps SIMD/NEON unsafe operations explicitly. - Adds a new fuzz workflow for PR/push smoke runs and scheduled/manual broader sweeps.
- Updates
rank_to_bucketvalidation, Python binding guards, tests, changelog, release notes, and threat model status.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
THREAT_MODEL.md |
Marks SIMD/fuzz/supply-chain mitigations as complete and updates workflow count. |
src/util.rs |
Wraps NEON popcount unsafe operations in explicit unsafe blocks. |
src/sign_bitmap.rs |
Wraps AVX-512 sign-bitmap kernels in explicit unsafe blocks. |
src/rank.rs |
Adds rank < d assertion, docs, and Rust regression test. |
src/quant_kernels.rs |
Wraps AVX2/AVX-512 quant kernels and documents the AVX2 helper exception. |
src/lib.rs |
Adds crate-wide #![deny(unsafe_op_in_unsafe_fn)]. |
src/fastscan.rs |
Removes now-redundant module-local unsafe lint. |
src/bitmap.rs |
Wraps AVX-512 bitmap kernels in explicit unsafe blocks. |
RELEASING.md |
Documents current immutable release and branch protection posture. |
ordvec-python/tests/test_redteam_fuzz.py |
Adds Python guard tests for invalid rank inputs. |
ordvec-python/src/lib.rs |
Mirrors new core rank guards as ValueErrors and refreshes pack comment. |
CHANGELOG.md |
Records fuzz CI, unsafe lint rollout, and rank behavior change. |
.github/workflows/fuzz.yml |
Adds bounded cargo-fuzz smoke and weekly sweep workflow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
bucket_ranks delegated the bits<=7 check to the per-entry rank_to_bucket call, so bucket_ranks(&[], bits>7) skipped it and silently returned an empty vec instead of failing loud. Assert bits<=7 up front so an invalid width is rejected for empty input too — matching the Python binding (which checks bits before its empty short-circuit) and making the documented panic contract accurate. Addresses a PR-review finding (copilot). Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
The fuzz.yml header said actions are SHA-pinned 'unlike the mutable @vn tags ci.yml uses' — but ci.yml is now fully SHA-pinned, so the comparison was wrong. Drop it; the comment states fuzz.yml's own pinning without an inaccurate baseline. Addresses a PR-review finding (copilot). Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
Pre-release closeout
Closes the final pre-release hardening items flagged as open in
THREAT_MODEL.md, plus therank_to_bucketconsistency wart. All four land here; the threat model andRELEASING.mdare updated to match the now-in-place state.1.
#![deny(unsafe_op_in_unsafe_fn)]crate-wide (THREAT-SIMD-001)Previously enforced only in
fastscan.rs. Moved the lint to the crate root (lib.rs); every unsafe op in thebitmap/sign_bitmap/quant_kernelsAVX-512/AVX2 kernels and theutilNEON popcount now sits in an explicitunsafe {}block. Purely additive — no kernel logic changed.horizontal_sum_avx2is intentionally left unwrapped: it is register-only (no memory access), so its intrinsics are safe under the#[target_feature]gate and an explicit block would beunused_unsafeunder-D warnings.2. Bounded CI fuzz smoke (THREAT-FUZZ-002)
New
.github/workflows/fuzz.yml:main: 60s/target smoke overload_rank,load_rankquant,fastscan_b2.cargo-fuzz is version-pinned (
0.13.1); actions SHA-pinned; read-only token; harden-runner on every job; the matrix target is passed viaenv:(norun:injection surface, per THREAT-CICD-001).3.
rank_to_bucketfail-loud (Closes #28)rank_to_bucketclamped withb.min(n_buckets - 1)and silently acceptedrank >= d. It now assertsrank < d(release too) and drops the clamp — mirroring the fail-loudpack_buckets/bucket_centrecontract (#26). Valid rank vectors (a permutation of[0, d)) are unaffected — verified: every caller feedsrank_transformoutput. The Python bindings gain matchingValueErrorguards onrank_to_bucketandbucket_ranksso the new assert never surfaces as aPanicException, with red-team tests for both.4. Stale binding comment
Refreshed the
pack_bucketsbinding comment: the core now asserts in-range codes (fails loud) rather than masking/truncating, so the comment's rationale was stale. Runtime behaviour was already safe (Python rejects bad codes first).Threat-model / RELEASING updates
mainbranch protection (PR review required; force-pushes and deletions blocked), closing the release-tag/asset mutability surface the registries already close. (Verified via API:mainprotection confirmed; immutable releases confirmed in the GitHub UI — the RESTimmutable_releasesfield and/rulesetsdon't surface that new feature, which is expected.)Verification (full local gate, all green)
Core:
fmt --check,clippy --all-targets --all-features -D warnings,test(default /--features experimental/--no-default-features),+1.89.0 build(MSRV),build --locked,RUSTFLAGS=-D warnings build,+nightly fuzz build,check --target aarch64-unknown-linux-gnu(NEON, force-recompiled).Binding:
fmt -p ordvec-python --check,clippy -p ordvec-python -D warnings,maturin develop+pytest→ 367 passed (incl. 2 new guard tests).Test plan
rank_to_bucket_rejects_rank_ge_d(#[should_panic]).test_rank_to_bucket_rank_ge_d_value_error,test_bucket_ranks_out_of_range_value_error.fuzz.ymlsmoke runs on this PR (first exercise of the workflow); actionlint/zizmor gate it.