Add v0.4 manifest sidecars and probe helpers

[Fieldnote-Echo]

Summary:

• make ordvec-manifest CLI optional behind the cli feature while keeping library defaults clap-free
• add create-time auxiliary artifacts, name-based load-plan lookup, and OrdinalDB ordinaldb.ids sidecar docs
• add Bitmap/SignBitmap swap_remove, typed validation helpers, and narrow RankQuant + SignBitmap two-stage helpers
• add first-publish recovery for ordvec-manifest crates.io byte-identity checks
• remediate bot review findings by rejecting whitespace-padded manifest aux names, trimming CLI aux paths, and returning an exact-length query error

Validation:

• cargo test -p ordvec
• cargo test -p ordvec-manifest --no-default-features
• cargo tree -p ordvec-manifest --no-default-features --edges normal (no clap)
• cargo test -p ordvec-manifest --features cli
• cargo test -p ordvec-manifest --features sqlite
• cargo test -p ordvec-manifest --all-features
• cargo fmt --check
• python3 tests/release_publish_invariants.py
• bash tests/release_signed_release_invariants.sh

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0)

1. Wheel artifact name collision ✓ Resolved 🐞 Bug ☼ Reliability

Description

In release.yml, manifest wheels are uploaded under the wheels-manifest-* prefix, which matches the
existing pattern: wheels-* artifact download used by the core pypi-canonical-dist job. If the
manifest wheel artifacts exist when that download step runs, core canonicalization may include
manifest wheels (causing filename-set verification failures or wrong inputs) depending on job
completion order.

Code

.github/workflows/release.yml[R590-594]

+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: wheels-manifest-${{ matrix.platform.runner }}-${{ matrix.platform.target }}
+          path: ordvec-manifest-python/dist/*.whl
+          if-no-files-found: error

Evidence

The manifest wheel artifact name wheels-manifest-... matches the existing core-wheel artifact
download glob wheels-*, so the core canonicalization job can unintentionally pull manifest wheels
when they’re available in the same workflow run.

.github/workflows/release.yml[590-594]
.github/workflows/release.yml[681-686]
.github/workflows/release.yml[727-733]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`build-manifest-wheels` uploads artifacts named `wheels-manifest-...`, but the core `pypi-canonical-dist` job downloads artifacts with `pattern: wheels-*`. Because artifact selection is workflow-wide and pattern-based, the core canonicalization step can unintentionally download manifest wheels when those artifacts have already been uploaded, making the release pipeline nondeterministic and potentially broken.

## Issue Context
- Core canonicalization downloads `pattern: wheels-*`.
- New manifest wheel artifacts are named `wheels-manifest-*` and therefore match that pattern.

## Fix Focus Areas
- .github/workflows/release.yml[590-594]
- .github/workflows/release.yml[727-733]

## Proposed fix
- Rename the manifest wheel artifact prefix to something that does **not** start with `wheels-` (e.g. `manifest-wheels-...`).
- Update `pypi-manifest-canonical-dist` to download the renamed artifacts (adjust its `pattern:` accordingly).
- (No changes needed to core `pypi-canonical-dist` once the manifest artifacts no longer match `wheels-*`.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Aux name not normalized 🐞 Bug ≡ Correctness

Description

VerifiedLoadPlan::auxiliary_by_name/require_auxiliary uses strict string equality on the stored
auxiliary artifact name, but manifest validation trims names only to check emptiness/duplication and
does not enforce that stored names are already trimmed. As a result, a manifest can verify
successfully yet require_auxiliary("ordinaldb.ids") fails if the manifest stored name has
leading/trailing whitespace.

Code

ordvec-manifest/src/lib.rs[R2846-2850]

+    pub fn auxiliary_by_name(&self, name: &str) -> Option<&VerifiedAuxiliaryArtifactPlan> {
+        self.auxiliary_artifacts
+            .iter()
+            .find(|artifact| artifact.name() == name)
+    }

Evidence

Validation trims names only for checks and does not reject whitespace-surrounded names; report/plans
preserve the raw name; lookup uses strict equality against that raw name. Therefore, a
whitespace-surrounded name can pass verification yet be unresolvable via the trimmed/canonical
string.

ordvec-manifest/src/lib.rs[497-518]
ordvec-manifest/src/lib.rs[1980-1994]
ordvec-manifest/src/lib.rs[2846-2850]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Name-based auxiliary lookup (`auxiliary_by_name` / `require_auxiliary`) compares names with strict equality, but manifest validation does not enforce canonical naming (e.g., no leading/trailing whitespace). This can produce surprising runtime failures: verification succeeds, yet lookup by canonical name fails.

### Issue Context
- Validation currently uses `artifact.name.trim()` for emptiness/dup checks, but preserves the original `artifact.name`.
- Report/plans preserve the original (possibly whitespace-surrounded) name.

### Fix Focus Areas
- ordvec-manifest/src/lib.rs[497-534]
- ordvec-manifest/src/lib.rs[1980-2004]
- ordvec-manifest/src/lib.rs[2846-2865]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Misleading length error 🐞 Bug ⚙ Maintainability

Description

RankQuant::try_search_with_sign_probe_with_policy returns OrdvecError::InvalidLength for a
single-query length mismatch, but OrdvecError::InvalidLength formats as “must be a multiple of dim”,
which is not the contract here (it must equal dim). This produces confusing diagnostics for a common
integration error (wrong query shape).

Code

src/quant.rs[R767-772]

+        if query.len() != self.dim {
+            return Err(OrdvecError::InvalidLength {
+                name: "query",
+                len: query.len(),
+                dim: self.dim,
+            });

Evidence

The new two-stage helper returns OrdvecError::InvalidLength when the query length is not exactly
dim, but the Display implementation for InvalidLength claims a “multiple of dim” requirement,
which does not match the helper’s equality check and will confuse users.

src/quant.rs[734-778]
src/lib.rs[114-139]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`RankQuant::try_search_with_sign_probe_with_policy` uses `OrdvecError::InvalidLength` when `query.len() != dim`, but `OrdvecError::InvalidLength`’s `Display` message states the length must be a *multiple* of `dim`, which is misleading for a single query vector.

### Issue Context
`OrdvecError::InvalidLength` is also used by `validate_flat_vectors_len`, where “multiple of dim” *is* correct; so the simplest targeted fix is to return a different error variant for the query-length mismatch.

### Fix Focus Areas
- src/quant.rs[755-773]
- src/lib.rs[114-147]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Previous review results

Review updated until commit 358e6b7

Results up to commit 7b64cde

🐞 Bugs (2) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0)

1. Aux name not normalized 🐞 Bug ≡ Correctness

Description

VerifiedLoadPlan::auxiliary_by_name/require_auxiliary uses strict string equality on the stored
auxiliary artifact name, but manifest validation trims names only to check emptiness/duplication and
does not enforce that stored names are already trimmed. As a result, a manifest can verify
successfully yet require_auxiliary("ordinaldb.ids") fails if the manifest stored name has
leading/trailing whitespace.

Code

ordvec-manifest/src/lib.rs[R2846-2850]

+    pub fn auxiliary_by_name(&self, name: &str) -> Option<&VerifiedAuxiliaryArtifactPlan> {
+        self.auxiliary_artifacts
+            .iter()
+            .find(|artifact| artifact.name() == name)
+    }

Evidence

Validation trims names only for checks and does not reject whitespace-surrounded names; report/plans
preserve the raw name; lookup uses strict equality against that raw name. Therefore, a
whitespace-surrounded name can pass verification yet be unresolvable via the trimmed/canonical
string.

ordvec-manifest/src/lib.rs[497-518]
ordvec-manifest/src/lib.rs[1980-1994]
ordvec-manifest/src/lib.rs[2846-2850]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Name-based auxiliary lookup (`auxiliary_by_name` / `require_auxiliary`) compares names with strict equality, but manifest validation does not enforce canonical naming (e.g., no leading/trailing whitespace). This can produce surprising runtime failures: verification succeeds, yet lookup by canonical name fails.

### Issue Context
- Validation currently uses `artifact.name.trim()` for emptiness/dup checks, but preserves the original `artifact.name`.
- Report/plans preserve the original (possibly whitespace-surrounded) name.

### Fix Focus Areas
- ordvec-manifest/src/lib.rs[497-534]
- ordvec-manifest/src/lib.rs[1980-2004]
- ordvec-manifest/src/lib.rs[2846-2865]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Misleading length error 🐞 Bug ⚙ Maintainability

Description

RankQuant::try_search_with_sign_probe_with_policy returns OrdvecError::InvalidLength for a
single-query length mismatch, but OrdvecError::InvalidLength formats as “must be a multiple of dim”,
which is not the contract here (it must equal dim). This produces confusing diagnostics for a common
integration error (wrong query shape).

Code

src/quant.rs[R767-772]

+        if query.len() != self.dim {
+            return Err(OrdvecError::InvalidLength {
+                name: "query",
+                len: query.len(),
+                dim: self.dim,
+            });

Evidence

The new two-stage helper returns OrdvecError::InvalidLength when the query length is not exactly
dim, but the Display implementation for InvalidLength claims a “multiple of dim” requirement,
which does not match the helper’s equality check and will confuse users.

src/quant.rs[734-778]
src/lib.rs[114-139]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`RankQuant::try_search_with_sign_probe_with_policy` uses `OrdvecError::InvalidLength` when `query.len() != dim`, but `OrdvecError::InvalidLength`’s `Display` message states the length must be a *multiple* of `dim`, which is misleading for a single query vector.

### Issue Context
`OrdvecError::InvalidLength` is also used by `validate_flat_vectors_len`, where “multiple of dim” *is* correct; so the simplest targeted fix is to return a different error variant for the query-length mismatch.

### Fix Focus Areas
- src/quant.rs[755-773]
- src/lib.rs[114-147]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Results up to commit 979d0a4

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0)

1. Wheel artifact name collision ✓ Resolved 🐞 Bug ☼ Reliability

Description

In release.yml, manifest wheels are uploaded under the wheels-manifest-* prefix, which matches the
existing pattern: wheels-* artifact download used by the core pypi-canonical-dist job. If the
manifest wheel artifacts exist when that download step runs, core canonicalization may include
manifest wheels (causing filename-set verification failures or wrong inputs) depending on job
completion order.

Code

.github/workflows/release.yml[R590-594]

+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: wheels-manifest-${{ matrix.platform.runner }}-${{ matrix.platform.target }}
+          path: ordvec-manifest-python/dist/*.whl
+          if-no-files-found: error

Evidence

The manifest wheel artifact name wheels-manifest-... matches the existing core-wheel artifact
download glob wheels-*, so the core canonicalization job can unintentionally pull manifest wheels
when they’re available in the same workflow run.

.github/workflows/release.yml[590-594]
.github/workflows/release.yml[681-686]
.github/workflows/release.yml[727-733]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`build-manifest-wheels` uploads artifacts named `wheels-manifest-...`, but the core `pypi-canonical-dist` job downloads artifacts with `pattern: wheels-*`. Because artifact selection is workflow-wide and pattern-based, the core canonicalization step can unintentionally download manifest wheels when those artifacts have already been uploaded, making the release pipeline nondeterministic and potentially broken.

## Issue Context
- Core canonicalization downloads `pattern: wheels-*`.
- New manifest wheel artifacts are named `wheels-manifest-*` and therefore match that pattern.

## Fix Focus Areas
- .github/workflows/release.yml[590-594]
- .github/workflows/release.yml[727-733]

## Proposed fix
- Rename the manifest wheel artifact prefix to something that does **not** start with `wheels-` (e.g. `manifest-wheels-...`).
- Update `pypi-manifest-canonical-dist` to download the renamed artifacts (adjust its `pattern:` accordingly).
- (No changes needed to core `pypi-canonical-dist` once the manifest artifacts no longer match `wheels-*`.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[qodo-code-review]

PR Summary by Qodo

Add manifest sidecar support, optional CLI feature, and two-stage probe helpers
✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

Walkthroughs

Description

• Make ordvec-manifest CLI optional via cli feature to keep the library clap-free
• Add named auxiliary sidecar declarations and load-plan lookup helpers for verified loading
• Add core ordvec validation, swap_remove support, and RankQuant+SignBitmap two-stage search helpers

Diagram

graph TD
A["ordvec-manifest (lib)"] --> B["create (aux sidecars)"] --> C["manifest.json + sidecars"]
A --> D["verify_for_load"] --> E["VerifiedLoadPlan"] --> H["Caller loader"]
F["ordvec (core)"] --> G["Two-stage search"] --> H

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Split into separate `ordvec-manifest-cli` crate

• ➕ Hard separation: the library can never accidentally pull CLI deps
• ➕ Cleaner feature matrix (no required-features binary gating needed)
• ➖ More crates to version/release in lockstep
• ➖ More duplicated glue code (args parsing, help text, install instructions)

2. Store auxiliary artifacts as a map keyed by name in the manifest

• ➕ O(1) lookup for auxiliary_by_name without scanning a Vec
• ➕ Uniqueness enforced structurally at serialization time
• ➖ Potentially less stable output ordering for human-diffing unless explicitly sorted
• ➖ Schema change impacts existing tooling and test fixtures

3. Use `Result`-based constructors instead of `validate_*` helpers

• ➕ Pushes correctness earlier (can't construct invalid indexes)
• ➕ Reduces runtime panics in new()/search() paths
• ➖ Breaking API change for existing callers relying on infallible constructors
• ➖ More pervasive refactor across constructors and loaders

Recommendation: The PR’s approach is a good incremental step: feature-gating the CLI keeps the library lightweight, auxiliary sidecars integrate into the existing manifest verification model, and validate_*/try_* helpers provide non-panicking paths without forcing a breaking constructor overhaul. If auxiliary artifacts grow significantly, consider a future schema evolution to a map keyed by name (with canonical sorting) to make lookup cheaper and uniqueness implicit.

File Changes

Enhancement (6)

lib.rs Add auxiliary artifact creation pipeline and load-plan name lookup +123/-1

Add auxiliary artifact creation pipeline and load-plan name lookup

• Introduces 'CreateAuxiliaryArtifact' and 'CreateManifestOptions::auxiliary_artifacts', hashing/validating named sidecars during manifest creation with limits and uniqueness checks. Adds 'VerifiedLoadPlan::auxiliary_by_name' and 'require_auxiliary', plus 'RequireAuxiliaryError' for ergonomic load-time enforcement.

ordvec-manifest/src/lib.rs

---

main.rs Extend CLI 'create' with '--aux' / '--optional-aux' sidecar flags +57/-3

Extend CLI 'create' with '--aux' / '--optional-aux' sidecar flags

• Adds clap parsing for 'NAME=PATH' auxiliary artifact args, splitting required and optional sidecars. Converts CLI inputs into 'CreateAuxiliaryArtifact' entries passed into 'CreateManifestOptions' so the CLI can declare sidecars while hashing them.

ordvec-manifest/src/main.rs

---

bitmap.rs Add 'Bitmap::swap_remove' and typed parameter validation +46/-1

Add 'Bitmap::swap_remove' and typed parameter validation

• Introduces 'Bitmap::validate_params' returning 'OrdvecError' for non-panicking validation. Adds 'swap_remove(idx)' to remove a vector by swapping in the last vector and truncating storage, returning the moved index.

src/bitmap.rs

---

lib.rs Introduce 'OrdvecError' and shared validation helpers; export two-stage policy +65/-1

Introduce 'OrdvecError' and shared validation helpers; export two-stage policy

• Adds a core 'OrdvecError' type and reusable validation helpers for flat vector lengths and candidate ID ranges. Exports 'TwoStageCandidatePolicy' from 'quant' for configuring two-stage probe behavior.

src/lib.rs

---

quant.rs Add RankQuant validation and SignBitmap-probed two-stage search helpers +137/-1

Add RankQuant validation and SignBitmap-probed two-stage search helpers

• Adds 'RankQuant::validate_params' to return structured errors rather than panicking. Introduces 'TwoStageCandidatePolicy' and 'try_search_with_sign_probe(_with_policy)' helpers that validate probe/query shape, enforce finiteness, select candidates via 'SignBitmap', validate candidate IDs, and then run subset scoring.

src/quant.rs

---

sign_bitmap.rs Add SignBitmap dim validation and 'swap_remove' removal primitive +41/-0

Add SignBitmap dim validation and 'swap_remove' removal primitive

• Introduces 'SignBitmap::validate_dim' returning 'OrdvecError' for consistent non-panicking parameter checks. Adds 'swap_remove(idx)' mirroring 'Bitmap' behavior to support in-place removal by swapping the last vector’s bitmap.

src/sign_bitmap.rs

---

Tests (4)

manifest.rs Add auxiliary artifact create/verify tests and gate CLI tests behind 'cli' +252/-36

Add auxiliary artifact create/verify tests and gate CLI tests behind 'cli'

• Adds coverage for sidecar declaration, name trimming, optional absence behavior, and 'require_auxiliary' error cases. Adds rejection tests for invalid/duplicated names, limit enforcement, missing files, and path policy violations; gates CLI-dependent tests with '#[cfg(feature = "cli")]' to keep no-default-features builds clap-free.

ordvec-manifest/tests/manifest.rs

---

bitmap.rs Test Bitmap swap_remove correctness and persistence invariants +64/-0

Test Bitmap swap_remove correctness and persistence invariants

• Adds test helpers to model corpus swapping behavior and verifies 'swap_remove' produces the same probe/search results as rebuilding from the expected corpus. Adds a persistence test ensuring write/load preserves probe behavior after swap_remove.

tests/index/bitmap.rs

---

two_stage.rs Add validation, swap_remove, and RankQuant+SignBitmap helper tests +158/-1

Add validation, swap_remove, and RankQuant+SignBitmap helper tests

• Adds tests for new core validation helpers, overflow-safe candidate policy clamping, SignBitmap swap_remove behavior, and persistence after removal. Verifies 'try_search_with_sign_probe_with_policy' matches manual candidate selection and validates error behavior for mismatched probes and invalid queries.

tests/index/two_stage.rs

---

release_publish_invariants.py Enforce release workflow includes ordvec-manifest recovery guard +54/-0

Enforce release workflow includes ordvec-manifest recovery guard

• Extends invariant checks to require exactly one manifest recovery step with the expected id and curl behavior. Verifies publish steps are ordered after recovery and conditionally skipped when crates.io already serves byte-identical bytes.

tests/release_publish_invariants.py

---

Documentation (2)

INDEX_PROVENANCE.md Document 'cli' feature usage and auxiliary sidecar loading guidance +11/-2

Document 'cli' feature usage and auxiliary sidecar loading guidance

• Updates example commands to enable the CLI via '--features cli'. Documents declaring sidecars at create time, requiring them before load, and provides OrdinalDB guidance for 'ordinaldb.ids' as a required auxiliary artifact.

docs/INDEX_PROVENANCE.md

---

README.md Update README for optional CLI and sidecar declarations +23/-3

Update README for optional CLI and sidecar declarations

• Updates install/run instructions to require '--features cli' for the binary. Adds examples and narrative for declaring sidecars ('--aux' / '--optional-aux') and requiring them via 'VerifiedLoadPlan::require_auxiliary', including OrdinalDB 'ordinaldb.ids' guidance.

ordvec-manifest/README.md

---

Other (2)

release.yml Add crates.io byte-identity recovery gate for ordvec-manifest publishes +31/-0

Add crates.io byte-identity recovery gate for ordvec-manifest publishes

• Adds a release step that checks whether crates.io already serves the target 'ordvec-manifest' .crate and verifies byte-identity against the SLSA-attested artifact. If already published and identical, publishing steps are skipped; if mismatched, the workflow fails to prevent unsafe recovery.

.github/workflows/release.yml

---

Cargo.toml Gate ordvec-manifest binary behind 'cli' feature and make clap optional +3/-1

Gate ordvec-manifest binary behind 'cli' feature and make clap optional

• Marks the 'ordvec-manifest' binary as 'required-features = ["cli"]' and makes 'clap' an optional dependency. Adds a 'cli' feature so the library default feature set stays empty and clap-free.

ordvec-manifest/Cargo.toml

---

[gemini-code-assist]

Code Review

This pull request introduces support for declaring and verifying auxiliary artifacts (sidecars) in the ordvec-manifest crate, including new CLI flags (--aux and --optional-aux) and programmatic APIs. It also adds validation helpers and swap_remove capabilities to the Bitmap and SignBitmap indices, alongside two-stage search helper methods in RankQuant. Feedback on the changes suggests trimming the parsed path string in the CLI argument parser to prevent leading or trailing whitespace from causing file-not-found errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Fieldnote-Echo]

/agentic_review

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code review by qodo was updated up to the latest commit 6516d6f

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code review by qodo was updated up to the latest commit 979d0a4

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code review by qodo was updated up to the latest commit b4e7a76

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code review by qodo was updated up to the latest commit 358e6b7