Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -814,7 +814,37 @@ Some use cases where `includeTags` and/or `excludeTags` may be more useful then



=== Usage in Re-Ranking Query
[[vector-reranking]]
== Usage in Re-Ranking Query

Dense vector similarity scores can be used to xref:query-guide:query-re-ranking.adoc[re-rank] first pass query results.
Possible use cases include:

* Re-ranking approximate results from a quantized vector field using full fidelity float vectors.
* Re-ranking lexical search results with dense vector similarity scores.

Details about using the ReRank Query Parser can be found in the xref:query-guide:query-re-ranking.adoc[Query Re-Ranking] section.

=== Re-Ranking with vectorSimilarity Function Query

The xref:query-guide:function-queries.adoc#vectorsimilarity-function[vectorSimilarity()] function can be used with the `{!func}` query parser to re-rank by vector similarity.
When used as a function query, `vectorSimilarity()` computes the exact similarity for only the candidate documents selected for re-ranking, without traversing the index graph.

Here is an example of re-ranking a lexical query using a `DenseVectorField` named `vector`:

[source,text]
?q=title:phone&rq={!rerank reRankQuery=$rqq reRankDocs=100 reRankWeight=1}&rqq={!func}vectorSimilarity(vector,[1.0,2.0,3.0,4.0])

NOTE: The default `reRankOperator` is `add`, which sums the first-pass score and the vector similarity score.
Since these scores may differ in magnitude, you can adjust `reRankWeight` to control the balance between them, or use `reRankOperator=replace` to score re-ranked documents by vector similarity alone.
Comment on lines +838 to +839

When using a quantized vector field type (such as `ScalarQuantizedDenseVectorField`), the KNN first pass scores are computed on the quantized vectors.
Here is an example of re-ranking those results with exact float similarity scores, where `topK` matches `reRankDocs`:

[source,text]
?q={!knn f=vector topK=100}[1.0,2.0,3.0,4.0]&rq={!rerank reRankQuery=$rqq reRankDocs=100 reRankWeight=1 reRankOperator=replace}&rqq={!func}vectorSimilarity(vector,[1.0,2.0,3.0,4.0])
Comment on lines +841 to +845

=== Re-Ranking with knn Query Parser

Both dense vector search query parsers can be used to rerank first pass query results:

Expand All @@ -832,8 +862,6 @@ the k-nearest neighbors(*in the whole index*) of the target vector to search.
This means the second pass `knn` is executed on the whole index anyway, which is a current limitation.

The final ranked list of results will have the first pass score(main query `q`) added to the second pass score(the approximated similarityFunction distance to the target vector to search) multiplied by a multiplicative factor(reRankWeight).

Details about using the ReRank Query Parser can be found in the xref:query-guide:query-re-ranking.adoc[Query Re-Ranking] section.
====


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,9 @@ In the example below, the scores for the top 1000 documents matching the query "
q=phone&rq={!rerank reRankQuery=$rqq reRankDocs=1000 reRankWeight=1 reRankOperator=replace}&rqq={!func v=div(1,sum(1,price))}
----

xref:query-guide:dense-vector-search.adoc[Dense vector fields] can also be used for re-ranking via the xref:query-guide:function-queries.adoc#vectorsimilarity-function[vectorSimilarity()] function query.
This computes exact vector similarity only for the re-ranked candidate documents. See xref:query-guide:dense-vector-search.adoc#vector-reranking[Usage in Re-Ranking Query] for examples and details.


=== LTR Query Parser

Expand Down
Loading