feat(query): add HybridQuery for FT.HYBRID server-side fusion#9
Conversation
Adds HybridQuery and SearchIndex.hybridSearch() that delegate text+vector
score fusion entirely to Redis via the FT.HYBRID command (introduced in
Redis OSS 8.4.0). Unlike Python redisvl's HybridQuery — which issues two
queries client-side and fuses ranks itself — this implementation runs as
a single round-trip with server-side RRF or LINEAR fusion.
The new method is separate from index.search() because FT.HYBRID has its
own command, options shape, and reply format. HybridSearchResult<T>
extends SearchResult<T> with executionTime and warnings fields.
Notable behaviours:
- text + textFieldName triggers tokenize + escape + OR-join. Omitting
textFieldName passes the body through verbatim so power users can use
full Redis Search syntax.
- vsimFilter is a raw string in the FT.SEARCH filter dialect (e.g.
'@brand:{nike}'). postFilter is a raw string in the FT.AGGREGATE
expression dialect (e.g. '@price < 200'). The two clauses use
different syntaxes server-side.
- LOAD always includes @__key so doc.id round-trips. Score aliases set
via YIELD_SCORE_AS are *not* added to LOAD — Redis already injects
them, and re-loading triggers "score alias already exists" errors.
- LOAD/SORTBY field references are auto-prefixed with @ when the user
passes bare names; explicit @ or $.path prefixes are preserved.
- Testcontainer image bumped from redis:8.0 to redis:8.4 for FT.HYBRID
support.
Marked @experimental in JSDoc since the underlying client.ft.hybrid()
is itself flagged experimental in @redis/search.
This change is intentionally self-contained: a tiny TokenEscaper and a
HybridTextScorer type are inlined into hybrid.ts so this PR can land
independently of the in-flight filter DSL work. A TODO at the top of
hybrid.ts tracks the cleanup commit that should follow once the filter
DSL merges (dedupe the helpers, widen vsimFilter to accept a typed
FilterExpression, drop HybridTextScorer in favour of the shared name).
Tests: 38 unit tests asserting toCommand() output for representative
configs (KNN/RANGE methods, RRF/LINEAR fusion, score aliases, LOAD
prefixing, SORTBY, NOSORT, postFilter, TIMEOUT) plus 7 integration tests
against a real Redis 8.4 Testcontainer covering each fusion method, each
vector method, vsimFilter, postFilter, verbatim text body, and LIMIT.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous matrix tested against redis/redis-stack-server tags
('latest' and '7.4.0-v8'), both of which are Redis Stack 7.x and don't
have FT.HYBRID (introduced in Redis 8.4). With the new HybridQuery
integration tests, CI would fail.
Switch to the official redis:* image — Redis 8 absorbed the Redis Stack
modules (search, JSON, time series, probabilistic) into the base image,
so a separate redis-stack-server is no longer needed. Matrix now tests
against '8.4' (minimum for FT.HYBRID, matching the Testcontainer
pinning) and 'latest'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts: # .github/workflows/test.yml
Covers the new HybridQuery surface: tokenised vs verbatim text body, KNN vs RANGE vector method, RRF vs LINEAR fusion, the two filter slots (vsimFilter in FT.SEARCH dialect, postFilter in FT.AGGREGATE dialect), score aliases, LOAD prefixing, and the HybridSearchResult return shape (executionTime + warnings). Calls out the Redis 8.4+ requirement and the @experimental status of client.ft.hybrid(). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts: # src/index.ts # src/indexes/search-index.ts # src/query/index.ts
… main Closes #14. Removes the inlined TokenEscaper, the HybridTextScorer alias, and the string-only vsimFilter that hybrid.ts carried so the feature could land independently of the filter DSL PR. With filter DSL now on main, swap to the canonical imports: - Drop the local TokenEscaper; import { TokenEscaper } from '../utils/token-escaper.js'. This also brings hybrid's tokenization in line with TextQuery's behaviour — wildcards (* and ?) are now escaped to literals by default rather than passed through. No existing test relied on the previous wildcard-preserving behaviour. - Drop type HybridTextScorer; use TextScorer from './text.js' instead. Removes the HybridTextScorer name from src/index.ts. - Widen vsimFilter from string to FilterInput so callers can pass either a typed FilterExpression (`Tag('brand').eq('nike')`) or a raw filter string. Route through renderFilter() from './base.js'. postFilter remains string-only — it uses the FT.AGGREGATE expression dialect, which FilterExpression doesn't render. Adds a unit test exercising FilterExpression as vsimFilter, and updates the hybrid-search docs to show the new typed form alongside the raw string form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Note: latest commit will close #20 |
|
Addressed the hybrid review pass in b7b087e.
Verification run locally:
|
ymendez-redis
left a comment
There was a problem hiding this comment.
Shall I add the search stopwords or is this not in scope yet?
Brings in #28 (AggregationQuery), #29 (RESP=3 guard), AGENTS.md, and the new contributing/aggregation docs. Additive resolutions: - src/index.ts: keep HybridQuery exports alongside AggregationQuery / Reducers from main. - src/query/index.ts: keep both re-exports. - src/indexes/search-index.ts: keep both the HybridQuery import and the AggregationQuery type import; keep hybridSearch() and the mapHybridRow helper, append aggregate() from main. - website/sidebars.ts: keep both `user-guide/hybrid-search` and `user-guide/aggregation` entries. No behavior changes — pure merge resolution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups from ymendez-redis's review on PR #9: - Drop the redundant `postFilter !== ''` check at the buildHybridOptions emit site. `assertNonEmptyString()` in the constructor already rejects empty (and whitespace-only) strings, so the second clause was dead. - Drop `m.k ?? 10` in encodeVectorMethod(). The constructor normalizes `vectorMethod.k` to `10` when undefined, so duplicating the default at the use site invites drift. Narrow the public type of HybridQuery.vectorMethod so KNN's `k` is `number` (not `number | undefined`), reflecting the post-constructor invariant and letting the use site read `m.k` directly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds first-class hybrid (text + vector) search support to the library by introducing a HybridQuery builder and SearchIndex.hybridSearch() that execute Redis’ FT.HYBRID for server-side fusion (RRF/LINEAR) in a single round trip, along with docs and CI/test updates for Redis 8.4 compatibility.
Changes:
- Introduce
HybridQuery(FT.HYBRIDcommand builder) andSearchIndex.hybridSearch()result mapping (HybridSearchResult). - Add unit + integration test coverage for command generation and Redis 8.4 execution.
- Add user-guide documentation + sidebar entry; update CI and local Testcontainers Redis image to support
FT.HYBRID.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| website/sidebars.ts | Adds the Hybrid Search page to the user guide sidebar. |
| website/docs/user-guide/hybrid-search.md | New Hybrid Search documentation and examples for HybridQuery / hybridSearch(). |
| tests/unit/query/hybrid.test.ts | Unit tests validating HybridQuery config validation and toCommand() output. |
| tests/integration/hybrid-search.test.ts | Integration tests against Redis 8.4+ for FT.HYBRID behavior. |
| tests/global-setup.ts | Bumps default Testcontainers Redis image to 8.4 (env-overridable). |
| src/query/index.ts | Re-exports the new hybrid query module. |
| src/query/hybrid.ts | Implements HybridQuery, validation, and FT.HYBRID option construction. |
| src/query/base.ts | Adds HybridSearchResult<T> interface. |
| src/indexes/search-index.ts | Adds hybridSearch() and FT.HYBRID row mapping to SearchDocument. |
| src/index.ts | Public exports for HybridQuery types and HybridSearchResult. |
| .github/workflows/test.yml | Skips hybrid integration tests in Redis Stack matrix and adds dedicated Redis 8.4 job. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /** Text scorer (defaults to server default `BM25STD`). */ | ||
| textScorer?: TextScorer; | ||
|
|
||
| /** Score fusion configuration. When omitted, the server default is used. */ |
| * convention. Anything that already starts with `@` or `$` is returned | ||
| * verbatim — the user is presumed to have supplied an explicit reference. | ||
| */ | ||
| function prefixFieldRef(name: string): string { | ||
| return name.startsWith('@') || name.startsWith('$') ? name : `@${name}`; |
| function validateFields(fields: string[] | undefined, label: string): string[] | undefined { | ||
| if (fields === undefined) return undefined; | ||
| return fields.map((field) => { | ||
| assertNonEmptyString(field, label); | ||
| return field; | ||
| }); | ||
| } | ||
|
|
||
| function validateSortBy( | ||
| sortBy: Array<{ field: string; direction?: 'ASC' | 'DESC' }> | undefined | ||
| ): Array<{ field: string; direction?: 'ASC' | 'DESC' }> | undefined { | ||
| if (sortBy === undefined) return undefined; | ||
| return sortBy.map((sort) => { | ||
| assertNonEmptyString(sort.field, 'sort field'); | ||
| if (sort.direction !== undefined && sort.direction !== 'ASC' && sort.direction !== 'DESC') { | ||
| throw new QueryValidationError('sort direction must be either ASC or DESC'); | ||
| } | ||
| return { ...sort }; | ||
| }); | ||
| } |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (1)
website/docs/user-guide/hybrid-search.md:272
- This snippet imports from
'redisvl', but the package name used throughout the docs/repo is'redis-vl'. Update the import so the example can be copied/pasted successfully.
```typescript
import { HybridQuery, HuggingFaceVectorizer } from 'redisvl';
| return `@${this.textFieldName}:(${tokens.join(' | ')})`; | ||
| } | ||
|
|
||
| private buildVsimClause(): FtHybridOptions['VSIM'] { | ||
| const vsim: FtHybridOptions['VSIM'] = { | ||
| field: `@${this.vectorField}`, |
Four changes from the Copilot review: - Docs: fix the package name in the Hybrid Search user guide. The examples imported from 'redisvl', but the package is published as 'redis-vl' (the rename landed in #21). Also fix the two JSDoc @example blocks in src/query/hybrid.ts and src/indexes/search-index.ts that the API-reference generator picks up. - Combine docstring: rewrite the comment on `HybridQueryConfig.combine`. The post-#9 hardening always emits a COMBINE clause (defaulting to RRF) so the combined score is yielded under a stable alias — the old "server default" wording was stale. - Strict prefixFieldRef: reject bare `$name` (no dot) the way AggregationQuery already does. In Redis Search `$name` is a PARAMS reference and has no business in a LOAD/SORTBY slot, so the previous pass-through behavior would silently produce invalid commands. Steer users toward '@name' (index field) or '$.name' (JSONPath). - Trim normalization: `validateFields`/`validateSortBy` enforce `trim() !== ''` but were keeping the original (potentially padded) string, which would produce `@ price ` in LOAD/SORTBY. Trim during normalization so the generated command is well-formed. Tests: 3 new — bare-$ rejection on returnFields and sortBy, plus a round-trip check that padded inputs are trimmed in both clauses. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds HybridQuery and SearchIndex.hybridSearch() that delegate text+vector score fusion entirely to Redis via the FT.HYBRID command (introduced in Redis OSS 8.4.0). Unlike Python redisvl's HybridQuery — which issues two queries client-side and fuses ranks itself — this implementation runs as a single round-trip with server-side RRF or LINEAR fusion.
The new method is separate from index.search() because FT.HYBRID has its own command, options shape, and reply format. HybridSearchResult extends SearchResult with executionTime and warnings fields.
Notable behaviours:
Marked @experimental in JSDoc since the underlying client.ft.hybrid() is itself flagged experimental in @redis/search.
Tests: 38 unit tests asserting toCommand() output for representative configs (KNN/RANGE methods, RRF/LINEAR fusion, score aliases, LOAD prefixing, SORTBY, NOSORT, postFilter, TIMEOUT) plus 7 integration tests against a real Redis 8.4 Testcontainer covering each fusion method, each vector method, vsimFilter, postFilter, verbatim text body, and LIMIT.