Skip to content

validator: add Alternator FLOAT32VECTOR type end-to-end tests#437

Merged
ewienik merged 1 commit into
scylladb:masterfrom
QuerthDP:alternator-v-type-tests
Jun 10, 2026
Merged

validator: add Alternator FLOAT32VECTOR type end-to-end tests#437
ewienik merged 1 commit into
scylladb:masterfrom
QuerthDP:alternator-v-type-tests

Conversation

@QuerthDP

@QuerthDP QuerthDP commented May 6, 2026

Copy link
Copy Markdown
Member

Add end-to-end coverage for the Alternator optimized vector type using requests with the new encoding injected into the JSON body.

Verify that queries work for items stored with the optimized vector representation and remain compatible with the existing L-based vector representation in the same index.

Fixes: VECTOR-666

@QuerthDP
QuerthDP force-pushed the alternator-v-type-tests branch from b9b620c to 7952f36 Compare May 27, 2026 10:51
@QuerthDP
QuerthDP force-pushed the alternator-v-type-tests branch 3 times, most recently from fdb2975 to 4dae248 Compare May 28, 2026 09:36
@QuerthDP
QuerthDP marked this pull request as ready for review May 28, 2026 09:38
@QuerthDP
QuerthDP requested review from ewienik and m-szymon May 28, 2026 09:38

@m-szymon m-szymon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I was considering adding such tests to types.rs tests (added here #467). I deliberately named it types, not key_types.
  2. Adding optimized data to index and querying are slightly separate issues.
  3. I was considering defining test datasets using "binary" type and only extending the json injector to convert them to FLOAT32VECTOR when expected - it could better reuse existing utils.

But it works, so consider if you would like to improve anything and we can merge it. I can later refactor.

Comment thread crates/validator/src/alternator/mod.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
@QuerthDP
QuerthDP force-pushed the alternator-v-type-tests branch from 4dae248 to 97c9968 Compare May 28, 2026 14:11
@QuerthDP

Copy link
Copy Markdown
Member Author

Changelog:

  • rebased on master
  • moved tests from query.rs to types.rs
  • merged the tests into one, which checks both insert and query

@QuerthDP
QuerthDP requested a review from m-szymon May 28, 2026 14:13
@m-szymon

Copy link
Copy Markdown
Collaborator

It is easier to ask AI to do it, than explain it: 3459c12

The problem is the pk_v_scan is not inserted at scan phase.
I was thinking about adding same trick as @nyh used for Java driver: scylladb/alternator-client-java#87

I also noticed that other test in that file don't actually check returned values. With that fixed, we could extend and use the query_with_key_type function.

@ewienik

ewienik commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

It is easier to ask AI to do it, than explain it: 3459c12

The problem is the pk_v_scan is not inserted at scan phase. I was thinking about adding same trick as @nyh used for Java driver: scylladb/alternator-client-java#87

I also noticed that other test in that file don't actually check returned values. With that fixed, we could extend and use the query_with_key_type function.

@m-szymon Does it mean that something needs to be done in this PR?

@m-szymon

m-szymon commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

It is easier to ask AI to do it, than explain it: 3459c12
The problem is the pk_v_scan is not inserted at scan phase. I was thinking about adding same trick as @nyh used for Java driver: scylladb/alternator-client-java#87
I also noticed that other test in that file don't actually check returned values. With that fixed, we could extend and use the query_with_key_type function.

@m-szymon Does it mean that something needs to be done in this PR?

Yes - one small inconsistency that pk_v_scan is not a scan test.
We can either remove/ignore it and enhance tests separately or fix from the beginning.

Add end-to-end coverage for the Alternator optimized vector type
using a magic-prefix binary encoding that is transparently converted
to FLOAT32VECTOR on the wire.

Verify that queries work for items stored with the optimized vector
representation and remain compatible with the existing L-based
vector representation in the same index.

Co-authored-by: Szymon Malewski <szymon.malewski@scylladb.com>
@QuerthDP
QuerthDP force-pushed the alternator-v-type-tests branch from 97c9968 to 2caaea1 Compare June 8, 2026 11:55
@QuerthDP

QuerthDP commented Jun 8, 2026

Copy link
Copy Markdown
Member Author

Changelog:

It looks fine to me. Added you as a co-author.

@m-szymon m-szymon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks fine to me. Added you as a co-author.

As a co-author my approve is not enough :)

@ewienik

@ewienik
ewienik added this pull request to the merge queue Jun 10, 2026
Merged via the queue into scylladb:master with commit 091a1c8 Jun 10, 2026
35 checks passed
@QuerthDP
QuerthDP deleted the alternator-v-type-tests branch July 16, 2026 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants