Skip to content

Pre-release closeout: crate-wide unsafe lint, CI fuzz smoke, rank_to_bucket fail-loud#72

Merged
Navi Bot (project-navi-bot) merged 7 commits into
mainfrom
chore/pre-release-closeout
May 26, 2026
Merged

Pre-release closeout: crate-wide unsafe lint, CI fuzz smoke, rank_to_bucket fail-loud#72
Navi Bot (project-navi-bot) merged 7 commits into
mainfrom
chore/pre-release-closeout

Conversation

@Fieldnote-Echo

Copy link
Copy Markdown
Member

Pre-release closeout

Closes the final pre-release hardening items flagged as open in THREAT_MODEL.md, plus the rank_to_bucket consistency wart. All four land here; the threat model and RELEASING.md are updated to match the now-in-place state.

1. #![deny(unsafe_op_in_unsafe_fn)] crate-wide (THREAT-SIMD-001)

Previously enforced only in fastscan.rs. Moved the lint to the crate root (lib.rs); every unsafe op in the bitmap / sign_bitmap / quant_kernels AVX-512/AVX2 kernels and the util NEON popcount now sits in an explicit unsafe {} block. Purely additive — no kernel logic changed. horizontal_sum_avx2 is intentionally left unwrapped: it is register-only (no memory access), so its intrinsics are safe under the #[target_feature] gate and an explicit block would be unused_unsafe under -D warnings.

2. Bounded CI fuzz smoke (THREAT-FUZZ-002)

New .github/workflows/fuzz.yml:

  • PR / push to main: 60s/target smoke over load_rank, load_rankquant, fastscan_b2.
  • Weekly (Thu 05:00 UTC) + manual: 300s/target sweep over all seven targets.

cargo-fuzz is version-pinned (0.13.1); actions SHA-pinned; read-only token; harden-runner on every job; the matrix target is passed via env: (no run: injection surface, per THREAT-CICD-001).

3. rank_to_bucket fail-loud (Closes #28)

rank_to_bucket clamped with b.min(n_buckets - 1) and silently accepted rank >= d. It now asserts rank < d (release too) and drops the clamp — mirroring the fail-loud pack_buckets / bucket_centre contract (#26). Valid rank vectors (a permutation of [0, d)) are unaffected — verified: every caller feeds rank_transform output. The Python bindings gain matching ValueError guards on rank_to_bucket and bucket_ranks so the new assert never surfaces as a PanicException, with red-team tests for both.

4. Stale binding comment

Refreshed the pack_buckets binding comment: the core now asserts in-range codes (fails loud) rather than masking/truncating, so the comment's rationale was stale. Runtime behaviour was already safe (Python rejects bad codes first).

Threat-model / RELEASING updates

  • SUPPLY-002 → mitigated: GitHub immutable releases enabled + main branch protection (PR review required; force-pushes and deletions blocked), closing the release-tag/asset mutability surface the registries already close. (Verified via API: main protection confirmed; immutable releases confirmed in the GitHub UI — the REST immutable_releases field and /rulesets don't surface that new feature, which is expected.)
  • SIMD-001 / FUZZ-002 → mitigated (this PR).
  • Corrected the workflow count (12 → 13).

Verification (full local gate, all green)

Core: fmt --check, clippy --all-targets --all-features -D warnings, test (default / --features experimental / --no-default-features), +1.89.0 build (MSRV), build --locked, RUSTFLAGS=-D warnings build, +nightly fuzz build, check --target aarch64-unknown-linux-gnu (NEON, force-recompiled).
Binding: fmt -p ordvec-python --check, clippy -p ordvec-python -D warnings, maturin develop + pytest367 passed (incl. 2 new guard tests).

Test plan

  • New Rust test rank_to_bucket_rejects_rank_ge_d (#[should_panic]).
  • New Python red-team tests test_rank_to_bucket_rank_ge_d_value_error, test_bucket_ranks_out_of_range_value_error.
  • CI: the new fuzz.yml smoke runs on this PR (first exercise of the workflow); actionlint/zizmor gate it.

Move #![deny(unsafe_op_in_unsafe_fn)] from fastscan.rs to the crate root
(lib.rs) so every unsafe operation in the SIMD kernels must sit in an
explicit `unsafe {}` block rather than leaning on an enclosing `unsafe fn`.
Wraps the AVX-512/AVX2 kernel bodies in bitmap.rs, sign_bitmap.rs and
quant_kernels.rs, and the NEON popcount in util.rs. horizontal_sum_avx2 is
register-only (no memory access), so its intrinsics are safe under the
#[target_feature] gate and need no block (an explicit one would be
unused_unsafe). Purely additive — no kernel logic changes. Keeps the unsafe
surface visible to future edits (THREAT-SIMD-001).

Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
Adds .github/workflows/fuzz.yml: a 60s/target smoke over load_rank,
load_rankquant and fastscan_b2 on every pull request / push to main, plus a
weekly full sweep over all seven targets. Surfaces loader / write->load
round-trip / FastScan-kernel regressions in CI between manual campaigns
(THREAT-FUZZ-002). cargo-fuzz is version-pinned (0.13.1) and every action is
SHA-pinned; read-only token; the matrix target is passed via env (no run:
injection surface, per THREAT-CICD-001).

Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
rank_to_bucket clamped the computed bucket with `b.min(n_buckets - 1)`,
silently accepting rank >= d. A valid rank is a position in [0, d), so it now
asserts `rank < d` and drops the clamp — matching the fail-loud contract of
the sibling primitives pack_buckets / bucket_centre (#26). Valid rank vectors
(a permutation of [0, d)) are unaffected.

The Python bindings gain matching guards on rank_to_bucket and bucket_ranks so
the new assert surfaces as a clean ValueError rather than a PanicException,
with red-team tests for both. Also refreshes the stale pack_buckets binding
comment (the core now asserts in-range, no longer masks).

Closes #28.

Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
Threat model + RELEASING now reflect the in-place controls:
- SUPPLY-002: GitHub immutable releases enabled + main branch protection
  (PR review required, force-pushes and deletions blocked), closing the
  release-tag/asset mutability surface the registries already close.
- SIMD-001: unsafe_op_in_unsafe_fn now denied crate-wide.
- FUZZ-002: fuzz.yml CI smoke + weekly sweep.
Corrects the workflow count (12 -> 13) and adds the CHANGELOG entries.

Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
@codecov

codecov Bot commented May 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 29.06574% with 205 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/bitmap.rs 0.00% 92 Missing ⚠️
src/sign_bitmap.rs 0.00% 58 Missing ⚠️
src/quant_kernels.rs 53.78% 55 Missing ⚠️

📢 Thoughts on this report? Let us know!

cargo-fuzz 0.13.1's bundled Cargo.lock pins rustix 0.36.x, which no longer compiles on current nightly (rustc_layout_scalar_valid_range_start is now a reserved attribute), so 'cargo install --locked' failed the three fuzz smoke jobs. Drop --locked; the tool stays version-pinned (0.13.1) and its build deps resolve to nightly-compatible versions. Verified locally on nightly.

Signed-off-by: Nelson Spence <nelson@projectnavi.ai>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request rolls out the #![deny(unsafe_op_in_unsafe_fn)] lint crate-wide, wrapping all unsafe operations in explicit unsafe {} blocks across the SIMD kernels (bitmap.rs, sign_bitmap.rs, quant_kernels.rs, and util.rs). It also updates rank_to_bucket to fail loud and panic when rank >= d instead of silently clamping, with corresponding clean ValueError handling added to the Python bindings. Additionally, the threat model, changelog, and releasing documentation have been updated to reflect these mitigations and the newly added CI fuzzing workflows. I have no feedback to provide as there are no review comments.

@Fieldnote-Echo

Copy link
Copy Markdown
Member Author

/agentic_review

@qodo-code-review

qodo-code-review Bot commented May 26, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Qodo Logo

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR closes pre-release hardening items by expanding unsafe-operation linting crate-wide, adding bounded fuzzing CI, and tightening rank_to_bucket/binding behavior to fail loudly on invalid ranks.

Changes:

  • Moves unsafe_op_in_unsafe_fn enforcement to the crate root and wraps SIMD/NEON unsafe operations explicitly.
  • Adds a new fuzz workflow for PR/push smoke runs and scheduled/manual broader sweeps.
  • Updates rank_to_bucket validation, Python binding guards, tests, changelog, release notes, and threat model status.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
THREAT_MODEL.md Marks SIMD/fuzz/supply-chain mitigations as complete and updates workflow count.
src/util.rs Wraps NEON popcount unsafe operations in explicit unsafe blocks.
src/sign_bitmap.rs Wraps AVX-512 sign-bitmap kernels in explicit unsafe blocks.
src/rank.rs Adds rank < d assertion, docs, and Rust regression test.
src/quant_kernels.rs Wraps AVX2/AVX-512 quant kernels and documents the AVX2 helper exception.
src/lib.rs Adds crate-wide #![deny(unsafe_op_in_unsafe_fn)].
src/fastscan.rs Removes now-redundant module-local unsafe lint.
src/bitmap.rs Wraps AVX-512 bitmap kernels in explicit unsafe blocks.
RELEASING.md Documents current immutable release and branch protection posture.
ordvec-python/tests/test_redteam_fuzz.py Adds Python guard tests for invalid rank inputs.
ordvec-python/src/lib.rs Mirrors new core rank guards as ValueErrors and refreshes pack comment.
CHANGELOG.md Records fuzz CI, unsafe lint rollout, and rank behavior change.
.github/workflows/fuzz.yml Adds bounded cargo-fuzz smoke and weekly sweep workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/rank.rs Outdated
Comment thread .github/workflows/fuzz.yml Outdated
bucket_ranks delegated the bits<=7 check to the per-entry rank_to_bucket call, so bucket_ranks(&[], bits>7) skipped it and silently returned an empty vec instead of failing loud. Assert bits<=7 up front so an invalid width is rejected for empty input too — matching the Python binding (which checks bits before its empty short-circuit) and making the documented panic contract accurate. Addresses a PR-review finding (copilot).

Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
The fuzz.yml header said actions are SHA-pinned 'unlike the mutable @vn tags ci.yml uses' — but ci.yml is now fully SHA-pinned, so the comparison was wrong. Drop it; the comment states fuzz.yml's own pinning without an inaccurate baseline. Addresses a PR-review finding (copilot).

Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
@project-navi-bot Navi Bot (project-navi-bot) merged commit e72a832 into main May 26, 2026
32 checks passed
@project-navi-bot Navi Bot (project-navi-bot) deleted the chore/pre-release-closeout branch May 26, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

consistency: rank_to_bucket clamps rank >= d instead of rejecting (mirror #26 bucket_centre fail-loud)

3 participants