[codex] polish 0.5 persistence and manifest release surfaces

README.md

@@ -516,7 +516,9 @@ floor from `is_multiple_of`). Because the kernels are built against those
 intrinsics, this is a hard compile floor, not just a convenience pin: a
 toolchain below 1.89 won't build the crate. Raising the MSRV is treated as a
 minor-version change under the
-[compatibility policy](docs/compatibility-policy.md).
+[compatibility policy](docs/compatibility-policy.md). The current feature
+stability matrix and downstream embedding notes live in
+[`docs/msrv-and-features.md`](docs/msrv-and-features.md).
 
 ## License
 

RELEASING.md

@@ -150,9 +150,16 @@ filename. Until a record is updated, the corresponding gated publish fails
 3. Bump the lockstep version (`Cargo.toml`,
    `ordvec-manifest/Cargo.toml` including its `ordvec` dependency,
    `ordvec-python/Cargo.toml`, `ordvec-python/pyproject.toml`,
-   `ordvec-python/python/ordvec/__init__.py`, and `ordvec-ffi/Cargo.toml`) and
-   update `CHANGELOG.md` with migration notes for every intentional
-   compatibility break. Commit on `main`.
+   `ordvec-python/python/ordvec/__init__.py`,
+   `ordvec-manifest-python/Cargo.toml`,
+   `ordvec-manifest-python/pyproject.toml`,
+   `ordvec-manifest-python/python/ordvec_manifest/__init__.py`, and
+   `ordvec-ffi/Cargo.toml`) and update `CHANGELOG.md` with migration notes for
+   every intentional compatibility break. Commit on `main`.
+   - Run `python tests/release_publish_invariants.py` after the bump; it checks
+     lockstep versions, MSRV/docs drift, registry metadata parity, Python
+     classifier/URL parity, docs.rs feature policy, package contents, and
+     release workflow invariants.
 4. Confirm CI is **green for current `main` HEAD**. `require-ci-green` checks
    `main` HEAD's SHA — which needs a **completed, successful** (not
    `cancelled`, not in-progress) run of `ci.yml`, `python.yml`, `fuzz.yml`,

docs/INDEX_PROVENANCE.md

@@ -67,9 +67,11 @@ descriptors, or make mutable shared storage immutable; callers still own the
 final policy decision and should load from the returned paths only while the
 verified files remain under their control.
 `ordvec-manifest/README.md` shows the intended verify-then-immediate-load
-pattern. If another process can mutate the manifest, index, row map, or sidecar
-between verification and load, re-run `verify_for_load` at the load boundary or
-load from immutable storage or a caller-owned loading path that pins bytes.
+pattern, a concrete `manifest.json + index.ovrq + ids.bin` sidecar-backed
+bundle, and the stable report fields/codes for sidecar audit logs. If another
+process can mutate the manifest, index, row map, or sidecar between
+verification and load, re-run `verify_for_load` at the load boundary or load
+from immutable storage or a caller-owned loading path that pins bytes.
 
 The manifest verifier checks:
 

docs/compatibility-policy.md

@@ -154,6 +154,8 @@ Deployment-side provenance guidance lives in
 The Rust MSRV is Rust 1.89. Raising it is a minor-version compatibility change
 and requires a reason in release notes. Keep `Cargo.toml` `rust-version`, the
 README MSRV badge/section, and the CI MSRV job synchronized.
+The release-facing feature matrix lives in
+[`msrv-and-features.md`](msrv-and-features.md).
 
 The core crate has no required system or numerical dependencies. Adding one, or
 adding an optional dependency feature that changes build expectations for

docs/msrv-and-features.md

@@ -0,0 +1,69 @@
+# MSRV and Feature Stability
+
+This matrix is the release-facing build contract for downstream embedders,
+packagers, and host systems. It complements the
+[pre-1.0 compatibility policy](compatibility-policy.md), which defines how
+compatibility-impacting changes are classified.
+
+Current MSRV: Rust 1.89.
+
+The MSRV applies to all Rust crates in this repository: `ordvec`,
+`ordvec-manifest`, `ordvec-python`, `ordvec-manifest-python`, and
+`ordvec-ffi`. The CI MSRV job, each `Cargo.toml` `rust-version`, and the
+README MSRV badge/section must stay synchronized. Raising the MSRV is a
+minor-version compatibility change and release notes must state the reason and
+any migration note.
+
+## Feature Matrix
+
+| Surface | Default features | Stable default-off features | Optional dependency features | Experimental/internal features |
+| --- | --- | --- | --- | --- |
+| `ordvec` | none | none | none | `experimental` exposes `MultiBucketBitmap`; `test-utils` is repo-test-only and has no public stability promise. |
+| `ordvec-manifest` | none | none | `cli`, `sqlite`, `sqlite-bundled` | none |
+| `ordvec-python` | n/a | n/a | n/a | n/a |
+| `ordvec-manifest-python` | n/a | n/a | n/a | n/a |
+| `ordvec-ffi` | none | none | none | none |
+
+SIMD dispatch in `ordvec` is not feature-gated. x86_64 dispatches AVX-512 and
+AVX2 at runtime, aarch64 uses NEON, wasm32 can use `simd128` when the target is
+built with that target feature, and other targets use the scalar fallback.
+Host systems should not need BLAS, LAPACK, `ndarray`, `faer`, or a native graph
+library to embed the core crate.
+
+`ordvec-manifest` keeps its library default feature set empty. The `cli`
+feature enables the `ordvec-manifest` binary and its `clap` dependency. The
+`sqlite` feature enables the local cache/audit subcommands; `sqlite-bundled`
+adds the bundled SQLite build through `rusqlite`.
+
+## Change Policy
+
+New feature flags must declare a stability class before merging:
+
+- stable default feature;
+- stable default-off feature;
+- optional dependency feature;
+- experimental/default-off feature;
+- internal repo-test-only feature.
+
+Changing the default feature set is compatibility-impacting and must be
+classified in release notes. Adding a new required system dependency, changing
+wheel platform expectations, or making an optional dependency effectively
+required is also compatibility-impacting.
+
+Experimental and internal features can change before 1.0, but releases should
+still call out changes likely to affect known downstream users. Stable feature
+changes should include examples or migration notes when the visible build or
+API surface changes.
+
+## Release Checks
+
+`python tests/release_publish_invariants.py` keeps the following in sync:
+
+- lockstep crate and Python package versions;
+- Rust MSRV declarations, README badge text, and CI MSRV toolchain;
+- crates.io metadata, PyPI metadata, docs.rs feature policy, and package
+  contents;
+- release workflow and registry preflight expectations.
+
+Release review should also compare touched code against this matrix so host
+systems can embed `ordvec` without hidden platform or feature surprises.

fuzz/fuzz_targets/load_bitmap.rs

@@ -1,15 +1,11 @@
 //! libFuzzer target for the `.ovbm` / `OVBM` loader (which also accepts the
 //! legacy `.tvbm` / `TVBM` magic), driven through the public
-//! `ordvec::Bitmap::load` entry point.
+//! `ordvec::Bitmap::load_from_bytes` entry point.
 //!
 //! The low-level `rank_io::load_bitmap` parser is crate-internal
-//! (`pub(crate)`), so the fuzzer exercises it through `Bitmap::load` — which
-//! runs that exact loader and then the type's post-load checks (the full
-//! public load path). `load` takes a `&Path`, and the only public load entry
-//! points are path-based (there is no public `&[u8]`/`Read` loader — issue
-//! #6), so a shared process-local scratch file (see [`scratch`]) feeds the
-//! loader the fuzz bytes without the per-iteration `mkstemp`/`unlink` churn a
-//! fresh `NamedTempFile` each run would incur.
+//! (`pub(crate)`), so the fuzzer exercises it through
+//! `Bitmap::load_from_bytes` — which runs that exact loader and then the
+//! type's post-load checks (the full public in-memory load path).
 //!
 //! Contract: on arbitrary bytes the loader must return `Ok(..)` or
 //! `Err(..)` — never panic, abort, or read out of bounds. libFuzzer
@@ -20,10 +16,6 @@
 
 use libfuzzer_sys::fuzz_target;
 
-mod scratch;
-
 fuzz_target!(|data: &[u8]| {
-    scratch::with_scratch_file(data, |path| {
-        let _ = ordvec::Bitmap::load(path);
-    });
+    let _ = ordvec::Bitmap::load_from_bytes(data);
 });

fuzz/fuzz_targets/load_fastscan.rs

@@ -1,13 +1,12 @@
 //! libFuzzer target for the `.ovfs` / `OVFS` loader (the FastScan b=2
 //! persistence format — new in the ordvec format, no legacy `TV*` magic),
-//! driven through the public `ordvec::RankQuantFastscan::load` entry point.
+//! driven through the public `ordvec::RankQuantFastscan::load_from_bytes`
+//! entry point.
 //!
 //! The low-level `rank_io::load_fastscan` parser is crate-internal
-//! (`pub(crate)`), so the fuzzer exercises it through `RankQuantFastscan::load`
-//! — which runs that exact loader (the full public load path). `load` takes a
-//! `&Path` and the only public load entry points are path-based (issue #6), so
-//! a shared process-local scratch file (see [`scratch`]) feeds the loader the
-//! fuzz bytes without per-iteration `mkstemp`/`unlink` churn.
+//! (`pub(crate)`), so the fuzzer exercises it through
+//! `RankQuantFastscan::load_from_bytes` — which runs that exact loader (the
+//! full public in-memory load path).
 //!
 //! Contract: on arbitrary bytes the loader must return `Ok(..)` or `Err(..)` —
 //! never panic, abort, or read out of bounds. libFuzzer treats any panic/abort
@@ -17,11 +16,7 @@
 
 use libfuzzer_sys::fuzz_target;
 
-mod scratch;
-
 fuzz_target!(|data: &[u8]| {
-    scratch::with_scratch_file(data, |path| {
-        // The only thing under test: arbitrary bytes -> Ok | Err, no panic.
-        let _ = ordvec::RankQuantFastscan::load(path);
-    });
+    // The only thing under test: arbitrary bytes -> Ok | Err, no panic.
+    let _ = ordvec::RankQuantFastscan::load_from_bytes(data);
 });

fuzz/fuzz_targets/load_rank.rs

@@ -1,15 +1,11 @@
 //! libFuzzer target for the `.ovr` / `OVR1` loader (which also accepts the
-//! legacy `.tvr` / `TVR1` magic), driven through the public `ordvec::Rank::load`
-//! entry point.
+//! legacy `.tvr` / `TVR1` magic), driven through the public
+//! `ordvec::Rank::load_from_bytes` entry point.
 //!
 //! The low-level `rank_io::load_rank` parser is crate-internal (`pub(crate)`),
-//! so the fuzzer exercises it through `Rank::load` — which runs that exact
-//! loader and then the type's post-load length check (the full public load
-//! path). `load` takes a `&Path`, and the only public load entry points are
-//! path-based (there is no public `&[u8]`/`Read` loader — issue #6), so a
-//! shared process-local scratch file (see [`scratch`]) feeds the loader the
-//! fuzz bytes without the per-iteration `mkstemp`/`unlink` churn a fresh
-//! `NamedTempFile` each run would incur.
+//! so the fuzzer exercises it through `Rank::load_from_bytes` — which runs
+//! that exact loader and then the type's post-load length check (the full
+//! public in-memory load path).
 //!
 //! Contract: on arbitrary bytes the loader must return `Ok(..)` or
 //! `Err(..)` — never panic, abort, or read out of bounds. libFuzzer
@@ -20,11 +16,7 @@
 
 use libfuzzer_sys::fuzz_target;
 
-mod scratch;
-
 fuzz_target!(|data: &[u8]| {
-    scratch::with_scratch_file(data, |path| {
-        // The only thing under test: arbitrary bytes -> Ok | Err, no panic.
-        let _ = ordvec::Rank::load(path);
-    });
+    // The only thing under test: arbitrary bytes -> Ok | Err, no panic.
+    let _ = ordvec::Rank::load_from_bytes(data);
 });

fuzz/fuzz_targets/load_rankquant.rs

@@ -1,15 +1,11 @@
 //! libFuzzer target for the `.ovrq` / `OVRQ` loader (which also accepts the
 //! legacy `.tvrq` / `TVRQ` magic), driven through the public
-//! `ordvec::RankQuant::load` entry point.
+//! `ordvec::RankQuant::load_from_bytes` entry point.
 //!
 //! The low-level `rank_io::load_rankquant` parser is crate-internal
-//! (`pub(crate)`), so the fuzzer exercises it through `RankQuant::load` —
-//! which runs that exact loader and then the type's post-load checks (the
-//! full public load path). `load` takes a `&Path`, and the only public load
-//! entry points are path-based (there is no public `&[u8]`/`Read` loader —
-//! issue #6), so a shared process-local scratch file (see [`scratch`]) feeds
-//! the loader the fuzz bytes without the per-iteration `mkstemp`/`unlink`
-//! churn a fresh `NamedTempFile` each run would incur.
+//! (`pub(crate)`), so the fuzzer exercises it through
+//! `RankQuant::load_from_bytes` — which runs that exact loader and then the
+//! type's post-load checks (the full public in-memory load path).
 //!
 //! Contract: on arbitrary bytes the loader must return `Ok(..)` or
 //! `Err(..)` — never panic, abort, or read out of bounds. libFuzzer
@@ -22,10 +18,6 @@
 
 use libfuzzer_sys::fuzz_target;
 
-mod scratch;
-
 fuzz_target!(|data: &[u8]| {
-    scratch::with_scratch_file(data, |path| {
-        let _ = ordvec::RankQuant::load(path);
-    });
+    let _ = ordvec::RankQuant::load_from_bytes(data);
 });

fuzz/fuzz_targets/load_sign_bitmap.rs

@@ -1,15 +1,11 @@
 //! libFuzzer target for the `.ovsb` / `OVSB` loader (which also accepts the
 //! legacy `.tvsb` / `TVSB` magic), driven through the public
-//! `ordvec::SignBitmap::load` entry point.
+//! `ordvec::SignBitmap::load_from_bytes` entry point.
 //!
 //! The low-level `rank_io::load_sign_bitmap` parser is crate-internal
-//! (`pub(crate)`), so the fuzzer exercises it through `SignBitmap::load` —
-//! which runs that exact loader and then the type's post-load checks (the full
-//! public load path). `load` takes a `&Path`, and the only public load entry
-//! points are path-based (there is no public `&[u8]`/`Read` loader — issue
-//! #6), so a shared process-local scratch file (see [`scratch`]) feeds the
-//! loader the fuzz bytes without the per-iteration `mkstemp`/`unlink` churn a
-//! fresh `NamedTempFile` each run would incur.
+//! (`pub(crate)`), so the fuzzer exercises it through
+//! `SignBitmap::load_from_bytes` — which runs that exact loader and then the
+//! type's post-load checks (the full public in-memory load path).
 //!
 //! Contract: on arbitrary bytes the loader must return `Ok(..)` or
 //! `Err(..)` — never panic, abort, or read out of bounds. libFuzzer
@@ -22,10 +18,6 @@
 
 use libfuzzer_sys::fuzz_target;
 
-mod scratch;
-
 fuzz_target!(|data: &[u8]| {
-    scratch::with_scratch_file(data, |path| {
-        let _ = ordvec::SignBitmap::load(path);
-    });
+    let _ = ordvec::SignBitmap::load_from_bytes(data);
 });