feat(astra): native hybrid + rerank via findAndRerank#59
Merged
Conversation
Wires astra-db-ts's `findAndRerank` through the Astra driver so
hybrid search against real Astra collections returns reranked hits
instead of 501.
## What changed
- **createCollection** now forwards `lexical` + `rerank` options to
the Data API when the descriptor opts in:
- `lexical.enabled: true` → Astra provisions a BM25-style
lexical index (analyzer optional)
- `reranking.enabled: true` + `provider` + `model` → Astra wires
a reranker service to the collection
Setting `reranking.enabled` without provider/model now throws
`WorkspaceMisconfiguredError` rather than silently dropping the
option.
- **searchHybrid** uses `coll.findAndRerank({}, { sort: { $hybrid:
{ $vector, $lexical } }, includeScores: true })`. Hits return
the `$reranker` score (or `$vector` / `$lexical` as a fallback).
- **Standalone rerank** is intentionally not exposed on Astra — the
Data API doesn't have a primitive to re-score an existing hit
set. Callers wanting rerank on Astra set `hybrid: true`.
- **Contract gating**: `searchHybrid` throws `NotSupportedError`
(→ 501) if the descriptor has `lexical.enabled: false` OR
`reranking.enabled: false`, with a clear message telling the
caller which flag to flip.
## Types
- `AstraCreateCollectionOptions` gains `lexical?` and `rerank?`
narrow shapes mirroring astra-db-ts's `CollectionLexicalOptions`
and `CollectionRerankOptions`.
- `AstraCollectionLike` gains `findAndRerank?(...)` with a
narrow `{ document, scores }` return type — enough to populate
`SearchHit`.
- New `AstraHybridSortObject` mirror of astra-db-ts's
`HybridSortObject`.
- New `rerankedToHit(row)` helper: score priority
`$reranker > $vector > $lexical`.
## Tests
- `FakeDb` / `FakeCollection` grow `findAndRerank` support gated
on the collection's lexical+rerank flags (captured at
`createCollection` time from the options the driver passes).
The fake scores docs with cosine on the vector side + token-
overlap on the lexical side, blended 50/50 as `$reranker`.
- 6 new tests in `tests/drivers/astra.test.ts`:
- create passes lexical + rerank through to Astra
- create throws when reranking is enabled without provider/model
- searchHybrid orders by the reranker score
- NotSupported when lexical disabled
- NotSupported when reranking disabled
- standalone rerank not exposed
Total: 449 tests (was 443; +6 astra hybrid).
## Wire contract
Unchanged. The 501 paths are the same errors; what changed is
which codepath inside the Astra driver produces them. `mock`
behavior is untouched. `lexicalWeight` stays on the request body
but Astra ignores it (documented in api-spec).
Second of three PRs in this Phase 2b-tail batch. UI catch-up follows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires astra-db-ts's `findAndRerank` through the Astra driver so hybrid search on real Astra collections returns reranked hits instead of 501.
Type additions
Test surface
`FakeDb` / `FakeCollection` grow `findAndRerank`, gated on `opts.lexical.enabled` + `opts.rerank.enabled` captured at `createCollection`. The fake scores via cosine(vector) + token-overlap(lexical), blended 50/50 as `$reranker` — faithful enough for ordering assertions. Six new tests cover create options, misconfiguration, the successful hybrid path, and both not-supported branches.
Test plan
Not in scope
Real-Astra integration tests — this PR validates the driver wiring against the fake. Enabling lexical + reranker on a real collection requires Astra-side service provisioning that varies by tenant; a gated CI job with creds can run the live path later.
`lexicalWeight` stays on the request body but Astra ignores it — the reranker owns the blend. Documented in api-spec.md.
Second of three PRs in this Phase 2b-tail batch. UI catch-up follows.