Add support for Scalar Quantization in HNSW.

[rahul-iyer]

Summary

This PR adds scalar quantization support for HNSW indexes and improves the cache_embeddings = true query path so quantized embeddings use a dense in-memory cache with transaction-aware invalidation.

The main behavioral goal is to keep quantized search on the fast cached path while preserving correctness around inserts, rollback, commit, and MVCC visibility.

Benchmark

Ran it on OpenAI 50k small

Results:

Variant	Load	Index Build	Ingest	Query Latency	DB Size	Bytes/Vec	Query RSS	Recall@10	Precision@10
base	5.862s	51.000s	56.862s	17.812 ms/query	606.5 MiB	12718.7	746.6 MiB	0.9960	0.9960
sq8	5.765s	13.111s	18.875s	16.366 ms/query	723.4 MiB	15170.2	327.1 MiB	0.9460	0.9460
sq16	5.421s	23.337s	28.758s	16.444 ms/query	908.0 MiB	19041.5	489.4 MiB	0.9950	0.9950

Speedup vs baseline:

Variant	Load	Index Build	Ingest	Query Latency	Query RSS Reduction
sq8	1.02x	3.89x	3.01x	1.09x	2.28x lower
sq16	1.08x	2.19x	1.98x	1.08x	1.53x lower

Storage overhead vs baseline:

Variant	DB Size	Bytes/Vec
sq8	1.19x	1.19x
sq16	1.50x	1.50x

Notes

• sq8 gives the largest build-time win and the lowest query RSS, with lower recall due to 8-bit quantization.
• sq16 keeps recall close to baseline while still improving index build, ingest, query latency, and query RSS.
• The benchmark uses cache_embeddings = true, which is the intended optimized path for quantized HNSW search in this PR.

[rahul-iyer]

Depends on LadybugDB/extensions#28