Skip to content

api: add optional composite SignBitmap + RankQuant two-stage index #192

@Fieldnote-Echo

Description

Context

A downstream database can own both SignBitmap and RankQuant, keep them in lockstep, and compose two-stage search manually. That is enough for OrdinalDB MVP. Longer-term, the composition is common and central enough that ordvec may want an optional composite type that owns the probe/body pair and exposes add/delete/search as one object.

This is wishlist-level and should build on the smaller primitives:

This is not the same as #170, which is about composite mixed-resolution RankQuant storage. This issue is specifically a SignBitmap probe plus RankQuant body wrapper.

Evidence

  • SignBitmap is already a compact sign probe with candidate APIs: src/sign_bitmap.rs:37, src/sign_bitmap.rs:133-210.
  • RankQuant is already the exact asymmetric rerank body with subset rerank: src/quant.rs:74, src/quant.rs:520-663.
  • The docs describe the two-stage path as bitmap/sign probe → RankQuant rerank: docs/RANK_MODES.md:35-43, docs/RANK_MODES.md:299-331.
  • Current public root surface exports primitives independently: src/lib.rs:63-67.

Proposed Shape

Sketch:

pub struct SignRankQuant {
    sign: SignBitmap,
    rank: RankQuant,
    options: TwoStageCandidatePolicy,
}

impl SignRankQuant {
    pub fn new(dim: usize, bits: u8) -> Self;
    pub fn add(&mut self, vectors: &[f32]);
    pub fn search(&self, queries: &[f32], k: usize) -> SearchResults;
    pub fn search_docset(&self, queries: &[f32], docset: DocsetView<'_>, k: usize) -> SearchResults;
    pub fn swap_remove(&mut self, idx: usize) -> usize;
}

Naming is flexible. It could also live behind an experimental or composite feature until the contract settles.

Acceptance Criteria

  • Owns SignBitmap and RankQuant and keeps lengths/dimensions in sync.
  • add mutates both indexes or neither, once fallible APIs exist.
  • search uses SignBitmap candidates followed by exact RankQuant rerank.
  • search_docset / allowlist path is supported once api: add docset-constrained SignBitmap to RankQuant two-stage search #188 lands, or explicitly deferred.
  • swap_remove updates both indexes consistently once api: add swap_remove to Bitmap and SignBitmap #186 lands.
  • Exposes accessors for len, is_empty, dim, bits, and byte-size accounting.
  • Tests compare results to manual composition and verify add/delete consistency.
  • Docs make clear this is still an embedded primitive, not a database, ID map, or persistence bundle.

Non-goals

  • No external IDs.
  • No metadata filters beyond caller-owned docsets.
  • No OrdinalDB .odb semantics.
  • No productization of RankQuantFastscan or hidden approximate scoring.

Metadata

Metadata

Assignees

No one assigned

    Labels

    core-apiCore search/index public API surface (pre-1.0)enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions