filter

[shaleenji]

No description provided.

[github-actions]

VectorDB Benchmark - Ready To Run

CI Passed ([lint + unit tests] (https://github.com/endee-io/endee/actions/runs/24508497817)) - benchmark options unlocked.

Post one of the command below. Only members with write access can trigger runs.

---

Available Modes

Mode	Command	What runs
Dense	/correctness_benchmarking dense	HNSW insert throughput · query P50/P95/P99 · recall@10 · concurrent QPS
Hybrid	/correctness_benchmarking hybrid	Dense + sparse BM25 fusion · same suite + fusion latency overhead

---

Infrastructure

Server	Role	Instance
Endee Server	Endee VectorDB — code from this branch	t2.large
Benchmark Server	Benchmark runner	t3a.large

Both servers start on demand and are always terminated after the run — pass or fail.

---

How Correctness Benchmarking Works

1. Post /correctness_benchmarking <mode>
2. Endee Server Create  →  this branch's code deployed  →  Endee starts in chosen mode
3. Benchmark Server Create  →  benchmark suite transferred
4. Benchmark Server runs correctness benchmarking against Endee Server
5. Results posted back here  →  pass/fail + full metrics table
6. Both servers terminated   →  always, even on failure

After a new push, CI must pass again before this menu reappears.

[shaleenji]

Oirginal Performance numbers for Int filter and label filter

[github-actions]

VectorDB Benchmark — — Failed

Triggered by @shaleenji · Commit ``

Step	Status
Provision Servers	Up
Deploy Endee Server	Done
Run Benchmark	Failed
Results	See reason below
Teardown	Done ✓

[shaleenji]

New Performance numbers for Int filter and label filter

[shaleenji]

1M indexing time on 8CPU, 30GB machine ovh B3-32 machine

int filter: 981 seconds
label filter: 1025 sec

[shaleenji]

The above code has two purposes:

1. Makes all the filter parsing first before actually inserting it to the DB
2. Performance enhancement by doing single MDBX transaction

1M indexing time on 8CPU, 30GB machine ovh B3-32 machine

int filter: 873 seconds (~11% reduction in indexing time)
label filter: 999 seconds

[github-actions]

VectorDB Benchmark - Ready To Run

CI Passed ([lint + unit tests] (https://github.com/endee-io/endee/actions/runs/25324862433)) - benchmark options unlocked.

Post one of the command below. Only members with write access can trigger runs.

---

Available Modes

Mode	Command	What runs
Dense	/correctness_benchmarking dense	HNSW insert throughput · query P50/P95/P99 · recall@10 · concurrent QPS
Hybrid	/correctness_benchmarking hybrid	Dense + sparse BM25 fusion · same suite + fusion latency overhead

---

Infrastructure

Server	Role	Instance
Endee Server	Endee VectorDB — code from this branch	t2.large
Benchmark Server	Benchmark runner	t3a.large

Both servers start on demand and are always terminated after the run — pass or fail.

---

How Correctness Benchmarking Works

1. Post /correctness_benchmarking <mode>
2. Endee Server Create  →  this branch's code deployed  →  Endee starts in chosen mode
3. Benchmark Server Create  →  benchmark suite transferred
4. Benchmark Server runs correctness benchmarking against Endee Server
5. Results posted back here  →  pass/fail + full metrics table
6. Both servers terminated   →  always, even on failure

After a new push, CI must pass again before this menu reappears.

[shaleenji]

Things todo:
Note: Key encoding is string-concatenated: field:value, field:id, and field:. Fields or values containing : can collide or break prefix scans. Length-prefixed or tuple-encoded MDBX keys would be safer.

[shaleenji]

Filter writes are not atomic with vector/meta/HNSW/WAL writes. A failure can leave vector metadata, filter indexes, sparse storage, and HNSW out of sync.