-
-
Notifications
You must be signed in to change notification settings - Fork 4
[codex] Document binding safety and release artifacts #249
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Navi Bot (project-navi-bot)
merged 3 commits into
main
from
codex/prerelease-bindings-artifact-docs
Jun 19, 2026
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # Artifact and Platform Matrix | ||
|
|
||
| This matrix is the release-facing inventory for what `ordvec` publishes, what | ||
| is repo-local, and which platform expectations are checked by the release | ||
| workflow. It complements `RELEASING.md` and `docs/msrv-and-features.md`; the | ||
| workflow remains the source of truth for the exact build jobs. The matrix | ||
| documents packaging and distribution compatibility for release verification. It | ||
| is not a service support commitment, runtime SLA, or guarantee that every host | ||
| environment matching a platform family is supported. | ||
|
|
||
| ## Published Artifacts | ||
|
|
||
| | Surface | Published where | Platform/build contract | Release verification | | ||
| | --- | --- | --- | --- | | ||
| | `ordvec` Rust crate | crates.io package `ordvec`; GitHub Release `.crate` asset | Rust 1.89 MSRV; default features empty; pure Rust, no BLAS/LAPACK/system numeric dependency | `cargo package --locked`; GitHub/Sigstore/SLSA provenance; pre-publish and post-publish byte identity against crates.io | | ||
| | `ordvec-manifest` Rust crate | crates.io package `ordvec-manifest`; GitHub Release `.crate` asset | Rust 1.89 MSRV; default features empty; optional `cli`, `sqlite`, and `sqlite-bundled` features | Built after matching `ordvec` exists; GitHub/Sigstore/SLSA provenance; byte identity against crates.io | | ||
| | Python `ordvec` | PyPI package `ordvec`; GitHub Release wheels and sdist | CPython 3.10+ abi3; `numpy>=2.2`; wheels for Linux x86_64 and Linux aarch64 are manylinux/glibc wheels; no musllinux/Alpine wheel is shipped yet; macOS aarch64 and Windows x64 wheels are also published; native extension modules are embedded in the wheel and do not load a separate `ordvec_ffi` library | Canonical wheel/sdist selection; linux/aarch64 native smoke; PyPI hash verification; PEP 740 attestation on fresh upload | | ||
| | Python `ordvec-manifest` | PyPI package `ordvec-manifest`; GitHub Release wheels and sdist | CPython 3.10+ abi3; Linux wheels are manylinux/glibc for x86_64 and aarch64; no musllinux/Alpine wheel is shipped yet; macOS aarch64 and Windows x64 wheels are also published; native extension modules are embedded in the wheel | Canonical wheel/sdist selection; linux/aarch64 native smoke; PyPI hash verification; PEP 740 attestation on fresh upload | | ||
| | Node/WASM | Not shipped; no npm package is published yet | Placeholder for issue #138; no JavaScript, TypeScript, or wasm package support is promised by this release | No release verification until a future packaging lane adds build jobs | | ||
| | JVM | Not shipped; no Maven/Gradle package is published yet | Placeholder for issue #139; no Java/Kotlin package support is promised by this release | No release verification until a future packaging lane adds build jobs | | ||
|
|
||
| The Python release currently expects exactly four wheels plus one sdist for | ||
| each Python package. There is no macOS x86_64 wheel leg in the current release | ||
| workflow. Linux users on musl-based distributions should build from source or | ||
| from the sdist unless a future release adds a `musllinux` wheel leg. | ||
|
|
||
| ## Repo-Local Sidecars | ||
|
|
||
| | Surface | Published where | Platform/build contract | Release role | | ||
| | --- | --- | --- | --- | | ||
| | `ordvec-ffi` | Not published to crates.io; built from the repository | Rust 1.89; emits `rlib`, `cdylib`, and `staticlib`; ABI v1 header is committed under `ordvec-ffi/include/`; cdylibs are named `libordvec_ffi.so`, `libordvec_ffi.dylib`, or `ordvec_ffi.dll`; static archives are named such as `libordvec_ffi.a` | C ABI compatibility surface for embedders; CI checks header drift and C link smoke; embedders must pair header and native library from the same git tag, require `ordvec_abi_version() == 1`, and compare `ordvec_version_string()` with the packaged native library | | ||
| | `ordvec-go` | Not published as a Go module release from this repo | Thin cgo wrapper over `ordvec-ffi`; links the local Rust library from the same git tag and ABI version | Binding smoke and race/cgocheck coverage for the C ABI contract; consumers must not mix Go wrapper, generated header, and native library from different tags | | ||
| | `benchmarks/beir-bench` | Not shipped in the published crate or wheels | Workspace benchmark crate with `publish = false` | Release-adjacent benchmark harness only; not a shipped user dependency | | ||
| | `fuzz/` | Not a workspace member and not published | `cargo-fuzz` crate with its own lockfile | Loader and parser hardening gate; release workflow runs smoke fuzz jobs | | ||
|
|
||
| Loading/linking strategy for the repo-local native sidecars is part of the | ||
| release contract: C embedders load or link the `ordvec-ffi` dynamic/static | ||
| library that matches the checked-in ABI v1 header; Go uses cgo to link that | ||
| same local `ordvec-ffi` build; Python wheels embed their own native extension | ||
| modules and do not load the repo-local `ordvec_ffi` shared library. Keep the | ||
| header, Go wrapper, and native libraries on the same git tag and ABI version. | ||
|
|
||
| ## Native Libraries and Version Alignment | ||
|
|
||
| - The Rust crates published to crates.io do not require a separately installed | ||
| native numeric library. | ||
| - The repo-local `ordvec-ffi` crate is named `ordvec_ffi`; dynamic builds emit | ||
| `libordvec_ffi.so` on Linux, `libordvec_ffi.dylib` on macOS, and | ||
| `ordvec_ffi.dll` on Windows. Static builds emit archives such as | ||
| `libordvec_ffi.a`. | ||
| - Python wheels embed their native extension modules inside the wheel. They do | ||
| not load a separately installed `ordvec_ffi` shared library. | ||
| - The Go wrapper links through cgo against a local `ordvec-ffi` build. Use the | ||
| Go package, generated header, and native library from the same git tag and ABI | ||
| version. | ||
| - C and Go embedders should check `ordvec_abi_version() == 1` and | ||
| `ordvec_version_string()` against the packaged native library. Do not mix | ||
| headers, Go wrappers, and native libraries from different tags. | ||
|
|
||
| ## SBOM Policy | ||
|
|
||
| The release workflow generates CycloneDX SBOMs for the Rust crate, manifest | ||
| crate, Python binding crate, and manifest Python binding crate as workflow | ||
| artifacts. Current PyPI distributions and GitHub Release assets do not embed or | ||
| attach those SBOM files. Published release assets are the canonical `.crate`, | ||
| wheel, and sdist files plus Sigstore and SLSA/in-toto provenance assets. | ||
|
|
||
| ## Platform Notes | ||
|
|
||
| - SIMD dispatch in the core crate is not feature-gated. x86_64 dispatches | ||
| AVX-512 and AVX2 at runtime where available, aarch64 uses NEON, wasm32 can | ||
| use `simd128` when built with that target feature, and unsupported targets | ||
| use scalar fallback paths. | ||
|
Fieldnote-Echo marked this conversation as resolved.
|
||
| - Native library naming, loading/linking strategy, and same-tag version | ||
| alignment are documented above in "Native Libraries and Version Alignment"; | ||
| those rules are part of this platform matrix, not optional packaging notes. | ||
| - Published Python wheels are abi3, so one wheel per platform covers CPython | ||
| 3.10 and newer for that platform. | ||
| - The release workflow keeps the GitHub Release draft until both Rust crates | ||
| and both Python packages have published successfully. A registry failure | ||
| leaves the release draft unpublished. | ||
| - GitHub Release assets include the canonical crate, wheel, and sdist files | ||
| plus provenance/attestation assets generated by the workflow. Verify | ||
| downloaded assets with `gh attestation verify` and registry-served hash | ||
| checks before treating them as deployment inputs. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| # Bindings Safety and Ownership | ||
|
|
||
| This is the cross-language contract for embedders using the Rust crate, Python | ||
| package, C ABI, or Go wrapper. It consolidates the binding notes that otherwise | ||
| live near each implementation. It does not add a new runtime policy: callers | ||
| still own scheduling, path trust, input mutability, and deployment provenance. | ||
|
|
||
| ## Concurrency | ||
|
|
||
| `ordvec` is read-concurrent and mutation-exclusive. | ||
|
|
||
| - Rust index values can be searched concurrently through shared references. | ||
| Mutation methods such as `add` require exclusive access. | ||
| - Python search, candidate-generation, scoring, and `add` methods release the | ||
| GIL while Rust performs the heavy work. PyO3 still enforces object borrow | ||
| rules, but caller-owned NumPy arrays are read in place while the GIL is | ||
| released. | ||
| - The C ABI permits concurrent `ordvec_index_search`, | ||
| `ordvec_index_probe`, and `ordvec_index_info` calls on one loaded handle. | ||
| `ordvec_index_free` must not race with any other call on that handle. | ||
| - The Go wrapper serializes `Close` against `Search` and `Info`; after | ||
| `Close`, both methods return `ErrClosed`. | ||
|
|
||
| ## Borrowed Inputs | ||
|
|
||
| Caller-provided buffers are borrowed for the duration of the call and are not | ||
| retained after the function returns. | ||
|
|
||
| - Do not mutate Rust slices, NumPy arrays, C buffers, or Go slices while a call | ||
| that received them is in progress. | ||
| - Query, corpus, candidate, output, hit, and stats buffers remain caller-owned | ||
| unless a specific API says otherwise. | ||
| - Candidate lists are entry lists, not sets. Duplicate candidate IDs are scored | ||
|
Fieldnote-Echo marked this conversation as resolved.
|
||
| independently, count toward candidate and vector-scored statistics, and can | ||
| produce duplicate hits. Deduplicate before calling when unique row IDs or | ||
| waste-free scoring matter. | ||
|
|
||
| ## Returned Memory | ||
|
|
||
| Current C ABI search calls do not return heap-owned result buffers. Callers | ||
| allocate and retain ownership of `hits_out`, `returned_out`, and `stats_out`. | ||
| `ordvec_index_load` returns an opaque handle that must be freed exactly once | ||
| with `ordvec_index_free` after all concurrent calls using that handle have | ||
| finished; ABI v1 has no general `ordvec_free` for result memory. | ||
|
|
||
| The Go wrapper copies C search hits into Go-owned slices. It frees temporary | ||
| `C.CString` values internally and releases the C index handle through `Close` | ||
| or its finalizer; Go callers do not free C result buffers. | ||
|
|
||
| ## Rows and External IDs | ||
|
|
||
| Core search results use internal row ordinals. The primitive persisted formats | ||
| do not carry an application ID map. | ||
|
|
||
| `ordvec-manifest` can bind an application-owned ID sidecar as a required | ||
| auxiliary artifact, but the primitive Rust, C, Go, and Python search paths still | ||
| return row ordinals. Host systems should maintain their own row-to-application | ||
| ID map and verify it together with the index when crossing a trust boundary. | ||
|
|
||
| ## Paths and Trust | ||
|
|
||
| `write` and `load` paths are trusted input. The core crate, Python binding, C | ||
| ABI, and Go wrapper forward paths to the filesystem without path traversal | ||
| sanitization or sandboxing. | ||
|
Fieldnote-Echo marked this conversation as resolved.
|
||
|
|
||
| Services that derive paths from user input should canonicalize and constrain | ||
| paths before calling `ordvec`, or use an application storage layer that never | ||
| exposes raw path choice to callers. Resolve paths against an allowed base | ||
| directory after symlink resolution, then reject any resolved path outside that | ||
| base. In Rust, use `std::fs::canonicalize`; in Python, use `pathlib.Path.resolve`; | ||
| in Go, combine lexical cleanup such as `filepath.Clean` with symlink-aware | ||
| resolution such as `filepath.EvalSymlinks`. For artifact integrity and sidecar | ||
| binding, use `ordvec-manifest`; it verifies hashes, declared metadata, | ||
| auxiliary artifacts, and attestation shape, but it does not sign files or | ||
| decide key policy. | ||
|
|
||
| ## Errors and Panics | ||
|
|
||
| - The Rust crate keeps fail-loud panicking constructors and methods where that | ||
| is the documented API. Existing `try_*` helpers return `OrdvecError` only | ||
| where explicitly provided. | ||
| - Python validates dimensions, dtypes, contiguity where required, finite | ||
| values, candidate ranges, and capacities at the boundary so common invalid | ||
| inputs raise typed Python exceptions instead of surfacing opaque Rust panics. | ||
| - The C ABI catches Rust panics at status-returning FFI boundaries and returns | ||
| `ORDVEC_STATUS_PANIC`; no Rust unwind crosses the ABI boundary. The same | ||
| thread's `ordvec_last_error()` is set to the panic payload when it is a string | ||
| or to fallback panic text otherwise. Successful status-returning calls clear | ||
| that thread-local error. Non-status helpers such as `ordvec_last_error()`, | ||
| version/status accessors, and `ordvec_index_free` do not report fallible | ||
| status. | ||
| - The Go wrapper maps C status values to Go errors and preserves the C ABI | ||
| pointer and lifetime rules. | ||
|
|
||
| ## Release Review Checklist | ||
|
|
||
| When a change touches a binding, review these questions before release: | ||
|
|
||
| - Does the change preserve read-concurrent, mutation-exclusive behavior? | ||
| - Are borrowed buffers still borrowed only for the documented call duration? | ||
| - Are path-trust assumptions unchanged or documented? | ||
| - Are row ordinals, duplicate candidates, and result shapes still described | ||
| consistently across Rust, Python, C, and Go? | ||
| - If a validation rule changes, is it a documented hardening fix rather than a | ||
| silent compatibility break? | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.