Skip to content

feat(query): add HybridQuery for FT.HYBRID server-side fusion#9

Merged
banker merged 11 commits into
mainfrom
feat/hybrid-search
May 22, 2026
Merged

feat(query): add HybridQuery for FT.HYBRID server-side fusion#9
banker merged 11 commits into
mainfrom
feat/hybrid-search

Conversation

@banker
Copy link
Copy Markdown
Contributor

@banker banker commented May 11, 2026

Adds HybridQuery and SearchIndex.hybridSearch() that delegate text+vector score fusion entirely to Redis via the FT.HYBRID command (introduced in Redis OSS 8.4.0). Unlike Python redisvl's HybridQuery — which issues two queries client-side and fuses ranks itself — this implementation runs as a single round-trip with server-side RRF or LINEAR fusion.

The new method is separate from index.search() because FT.HYBRID has its own command, options shape, and reply format. HybridSearchResult extends SearchResult with executionTime and warnings fields.

Notable behaviours:

  • text + textFieldName triggers tokenize + escape + OR-join. Omitting textFieldName passes the body through verbatim so power users can use full Redis Search syntax.
  • vsimFilter is a raw string in the FT.SEARCH filter dialect (e.g. '@brand:{nike}'). postFilter is a raw string in the FT.AGGREGATE expression dialect (e.g. '@price < 200'). The two clauses use different syntaxes server-side.
  • LOAD always includes @__key so doc.id round-trips. Score aliases set via YIELD_SCORE_AS are not added to LOAD — Redis already injects them, and re-loading triggers "score alias already exists" errors.
  • LOAD/SORTBY field references are auto-prefixed with @ when the user passes bare names; explicit @ or $.path prefixes are preserved.
  • Testcontainer image bumped from redis:8.0 to redis:8.4 for FT.HYBRID support.

Marked @experimental in JSDoc since the underlying client.ft.hybrid() is itself flagged experimental in @redis/search.

Tests: 38 unit tests asserting toCommand() output for representative configs (KNN/RANGE methods, RRF/LINEAR fusion, score aliases, LOAD prefixing, SORTBY, NOSORT, postFilter, TIMEOUT) plus 7 integration tests against a real Redis 8.4 Testcontainer covering each fusion method, each vector method, vsimFilter, postFilter, verbatim text body, and LIMIT.

Adds HybridQuery and SearchIndex.hybridSearch() that delegate text+vector
score fusion entirely to Redis via the FT.HYBRID command (introduced in
Redis OSS 8.4.0). Unlike Python redisvl's HybridQuery — which issues two
queries client-side and fuses ranks itself — this implementation runs as
a single round-trip with server-side RRF or LINEAR fusion.

The new method is separate from index.search() because FT.HYBRID has its
own command, options shape, and reply format. HybridSearchResult<T>
extends SearchResult<T> with executionTime and warnings fields.

Notable behaviours:

- text + textFieldName triggers tokenize + escape + OR-join. Omitting
  textFieldName passes the body through verbatim so power users can use
  full Redis Search syntax.
- vsimFilter is a raw string in the FT.SEARCH filter dialect (e.g.
  '@brand:{nike}'). postFilter is a raw string in the FT.AGGREGATE
  expression dialect (e.g. '@price < 200'). The two clauses use
  different syntaxes server-side.
- LOAD always includes @__key so doc.id round-trips. Score aliases set
  via YIELD_SCORE_AS are *not* added to LOAD — Redis already injects
  them, and re-loading triggers "score alias already exists" errors.
- LOAD/SORTBY field references are auto-prefixed with @ when the user
  passes bare names; explicit @ or $.path prefixes are preserved.
- Testcontainer image bumped from redis:8.0 to redis:8.4 for FT.HYBRID
  support.

Marked @experimental in JSDoc since the underlying client.ft.hybrid()
is itself flagged experimental in @redis/search.

This change is intentionally self-contained: a tiny TokenEscaper and a
HybridTextScorer type are inlined into hybrid.ts so this PR can land
independently of the in-flight filter DSL work. A TODO at the top of
hybrid.ts tracks the cleanup commit that should follow once the filter
DSL merges (dedupe the helpers, widen vsimFilter to accept a typed
FilterExpression, drop HybridTextScorer in favour of the shared name).

Tests: 38 unit tests asserting toCommand() output for representative
configs (KNN/RANGE methods, RRF/LINEAR fusion, score aliases, LOAD
prefixing, SORTBY, NOSORT, postFilter, TIMEOUT) plus 7 integration tests
against a real Redis 8.4 Testcontainer covering each fusion method, each
vector method, vsimFilter, postFilter, verbatim text body, and LIMIT.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@banker banker requested a review from booleanhunter May 11, 2026 13:52
@banker banker marked this pull request as draft May 11, 2026 13:56
The previous matrix tested against redis/redis-stack-server tags
('latest' and '7.4.0-v8'), both of which are Redis Stack 7.x and don't
have FT.HYBRID (introduced in Redis 8.4). With the new HybridQuery
integration tests, CI would fail.

Switch to the official redis:* image — Redis 8 absorbed the Redis Stack
modules (search, JSON, time series, probabilistic) into the base image,
so a separate redis-stack-server is no longer needed. Matrix now tests
against '8.4' (minimum for FT.HYBRID, matching the Testcontainer
pinning) and 'latest'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@banker banker marked this pull request as ready for review May 12, 2026 13:15
banker and others added 4 commits May 12, 2026 09:19
# Conflicts:
#	.github/workflows/test.yml
Covers the new HybridQuery surface: tokenised vs verbatim text body,
KNN vs RANGE vector method, RRF vs LINEAR fusion, the two filter slots
(vsimFilter in FT.SEARCH dialect, postFilter in FT.AGGREGATE dialect),
score aliases, LOAD prefixing, and the HybridSearchResult return shape
(executionTime + warnings).

Calls out the Redis 8.4+ requirement and the @experimental status of
client.ft.hybrid().

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts:
#	src/index.ts
#	src/indexes/search-index.ts
#	src/query/index.ts
… main

Closes #14.

Removes the inlined TokenEscaper, the HybridTextScorer alias, and the
string-only vsimFilter that hybrid.ts carried so the feature could land
independently of the filter DSL PR. With filter DSL now on main, swap to
the canonical imports:

- Drop the local TokenEscaper; import { TokenEscaper } from
  '../utils/token-escaper.js'. This also brings hybrid's tokenization in
  line with TextQuery's behaviour — wildcards (* and ?) are now escaped
  to literals by default rather than passed through. No existing test
  relied on the previous wildcard-preserving behaviour.
- Drop type HybridTextScorer; use TextScorer from './text.js' instead.
  Removes the HybridTextScorer name from src/index.ts.
- Widen vsimFilter from string to FilterInput so callers can pass either
  a typed FilterExpression (`Tag('brand').eq('nike')`) or a raw filter
  string. Route through renderFilter() from './base.js'.

postFilter remains string-only — it uses the FT.AGGREGATE expression
dialect, which FilterExpression doesn't render.

Adds a unit test exercising FilterExpression as vsimFilter, and updates
the hybrid-search docs to show the new typed form alongside the raw
string form.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@banker
Copy link
Copy Markdown
Contributor Author

banker commented May 13, 2026

Note: latest commit will close #20

@nkanu17
Copy link
Copy Markdown
Contributor

nkanu17 commented May 13, 2026

Addressed the hybrid review pass in b7b087e.
Changes:

  • Always emits a default RRF COMBINE with YIELD_SCORE_AS so default HybridQuery results have a stable combined score alias.
  • Keeps result mapping defensive by falling back to Redis default score keys if present.
  • Adds runtime validation for numResults, offset, timeout, textFieldName, returnFields, sortBy, and noSort/sortBy conflicts.
  • Restores the main CI matrix to Redis Stack latest + 7.4.0-v8, and adds a separate Redis 8.4 HybridQuery job for FT.HYBRID coverage.

Verification run locally:

  • npm run type-check
  • npm run test:unit
  • npm test -- tests/integration/hybrid-search.test.ts
  • REDISVL_SKIP_HYBRID=true npm test -- tests/integration/hybrid-search.test.ts
  • npm run build

Copy link
Copy Markdown
Collaborator

@ymendez-redis ymendez-redis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall I add the search stopwords or is this not in scope yet?

Comment thread src/query/hybrid.ts
Comment thread src/query/hybrid.ts Outdated
Comment thread src/query/hybrid.ts Outdated
banker and others added 2 commits May 22, 2026 11:30
Brings in #28 (AggregationQuery), #29 (RESP=3 guard), AGENTS.md, and
the new contributing/aggregation docs.

Additive resolutions:

- src/index.ts: keep HybridQuery exports alongside AggregationQuery /
  Reducers from main.
- src/query/index.ts: keep both re-exports.
- src/indexes/search-index.ts: keep both the HybridQuery import and
  the AggregationQuery type import; keep hybridSearch() and the
  mapHybridRow helper, append aggregate() from main.
- website/sidebars.ts: keep both `user-guide/hybrid-search` and
  `user-guide/aggregation` entries.

No behavior changes — pure merge resolution.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups from ymendez-redis's review on PR #9:

- Drop the redundant `postFilter !== ''` check at the buildHybridOptions
  emit site. `assertNonEmptyString()` in the constructor already rejects
  empty (and whitespace-only) strings, so the second clause was dead.

- Drop `m.k ?? 10` in encodeVectorMethod(). The constructor normalizes
  `vectorMethod.k` to `10` when undefined, so duplicating the default at
  the use site invites drift. Narrow the public type of
  HybridQuery.vectorMethod so KNN's `k` is `number` (not `number | undefined`),
  reflecting the post-constructor invariant and letting the use site read
  `m.k` directly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 22, 2026 17:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class hybrid (text + vector) search support to the library by introducing a HybridQuery builder and SearchIndex.hybridSearch() that execute Redis’ FT.HYBRID for server-side fusion (RRF/LINEAR) in a single round trip, along with docs and CI/test updates for Redis 8.4 compatibility.

Changes:

  • Introduce HybridQuery (FT.HYBRID command builder) and SearchIndex.hybridSearch() result mapping (HybridSearchResult).
  • Add unit + integration test coverage for command generation and Redis 8.4 execution.
  • Add user-guide documentation + sidebar entry; update CI and local Testcontainers Redis image to support FT.HYBRID.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
website/sidebars.ts Adds the Hybrid Search page to the user guide sidebar.
website/docs/user-guide/hybrid-search.md New Hybrid Search documentation and examples for HybridQuery / hybridSearch().
tests/unit/query/hybrid.test.ts Unit tests validating HybridQuery config validation and toCommand() output.
tests/integration/hybrid-search.test.ts Integration tests against Redis 8.4+ for FT.HYBRID behavior.
tests/global-setup.ts Bumps default Testcontainers Redis image to 8.4 (env-overridable).
src/query/index.ts Re-exports the new hybrid query module.
src/query/hybrid.ts Implements HybridQuery, validation, and FT.HYBRID option construction.
src/query/base.ts Adds HybridSearchResult<T> interface.
src/indexes/search-index.ts Adds hybridSearch() and FT.HYBRID row mapping to SearchDocument.
src/index.ts Public exports for HybridQuery types and HybridSearchResult.
.github/workflows/test.yml Skips hybrid integration tests in Redis Stack matrix and adds dedicated Redis 8.4 job.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread website/docs/user-guide/hybrid-search.md Outdated
Comment thread src/query/hybrid.ts Outdated
/** Text scorer (defaults to server default `BM25STD`). */
textScorer?: TextScorer;

/** Score fusion configuration. When omitted, the server default is used. */
Comment thread src/query/hybrid.ts Outdated
Comment on lines +147 to +151
* convention. Anything that already starts with `@` or `$` is returned
* verbatim — the user is presumed to have supplied an explicit reference.
*/
function prefixFieldRef(name: string): string {
return name.startsWith('@') || name.startsWith('$') ? name : `@${name}`;
Comment thread src/query/hybrid.ts
Comment on lines +184 to +203
function validateFields(fields: string[] | undefined, label: string): string[] | undefined {
if (fields === undefined) return undefined;
return fields.map((field) => {
assertNonEmptyString(field, label);
return field;
});
}

function validateSortBy(
sortBy: Array<{ field: string; direction?: 'ASC' | 'DESC' }> | undefined
): Array<{ field: string; direction?: 'ASC' | 'DESC' }> | undefined {
if (sortBy === undefined) return undefined;
return sortBy.map((sort) => {
assertNonEmptyString(sort.field, 'sort field');
if (sort.direction !== undefined && sort.direction !== 'ASC' && sort.direction !== 'DESC') {
throw new QueryValidationError('sort direction must be either ASC or DESC');
}
return { ...sort };
});
}
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 22, 2026 19:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

website/docs/user-guide/hybrid-search.md:272

  • This snippet imports from 'redisvl', but the package name used throughout the docs/repo is 'redis-vl'. Update the import so the example can be copied/pasted successfully.
```typescript
import { HybridQuery, HuggingFaceVectorizer } from 'redisvl';

Comment thread src/query/hybrid.ts
Comment on lines +407 to +412
return `@${this.textFieldName}:(${tokens.join(' | ')})`;
}

private buildVsimClause(): FtHybridOptions['VSIM'] {
const vsim: FtHybridOptions['VSIM'] = {
field: `@${this.vectorField}`,
Comment thread src/query/hybrid.ts Outdated
Comment thread src/query/hybrid.ts Outdated
Comment thread src/indexes/search-index.ts Outdated
Comment thread website/docs/user-guide/hybrid-search.md Outdated
Four changes from the Copilot review:

- Docs: fix the package name in the Hybrid Search user guide. The
  examples imported from 'redisvl', but the package is published as
  'redis-vl' (the rename landed in #21). Also fix the two JSDoc
  @example blocks in src/query/hybrid.ts and src/indexes/search-index.ts
  that the API-reference generator picks up.

- Combine docstring: rewrite the comment on `HybridQueryConfig.combine`.
  The post-#9 hardening always emits a COMBINE clause (defaulting to
  RRF) so the combined score is yielded under a stable alias — the old
  "server default" wording was stale.

- Strict prefixFieldRef: reject bare `$name` (no dot) the way
  AggregationQuery already does. In Redis Search `$name` is a PARAMS
  reference and has no business in a LOAD/SORTBY slot, so the previous
  pass-through behavior would silently produce invalid commands. Steer
  users toward '@name' (index field) or '$.name' (JSONPath).

- Trim normalization: `validateFields`/`validateSortBy` enforce
  `trim() !== ''` but were keeping the original (potentially padded)
  string, which would produce `@ price ` in LOAD/SORTBY. Trim during
  normalization so the generated command is well-formed.

Tests: 3 new — bare-$ rejection on returnFields and sortBy, plus a
round-trip check that padded inputs are trimmed in both clauses.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@banker banker merged commit 34c3ff7 into main May 22, 2026
10 checks passed
@banker banker deleted the feat/hybrid-search branch May 22, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants