Skip to content

fix: optimize segment compaction by reusing first vector index#114

Open
kgeg401 wants to merge 2 commits intoalibaba:mainfrom
kgeg401:fix/issue-98-optimize-merge-reuse
Open

fix: optimize segment compaction by reusing first vector index#114
kgeg401 wants to merge 2 commits intoalibaba:mainfrom
kgeg401:fix/issue-98-optimize-merge-reuse

Conversation

@kgeg401
Copy link

@kgeg401 kgeg401 commented Feb 14, 2026

Summary

  • add a no-filter compaction fast path that reuses the first segment's vector index as merge base
  • keep full rebuild behavior as fallback for filtered compaction and unsupported index layouts
  • cover both vector-index and filtered fallback regressions with segment helper tests

Root Cause

ReduceVectorIndex always rebuilt merged indexes from scratch, even when compaction had no filter and segments were layout-compatible. That caused avoidable index rebuild work during merge/compact.

Validation

  • Added CompactTask_VectorIndexThreeSegmentsRegression to verify no-filter multi-segment compaction correctness
  • Added CompactTask_FilterMultiSegmentsRegression to verify filtered path still uses fallback semantics
  • Local compile/test execution is not available in this environment (no C/C++ compiler toolchain), so CI is the authoritative validation

Fixes #98

@feihongxu0824
Copy link
Collaborator

Thank you for your contribution. The zvec maintainer team is currently on holiday for the Chinese New Year (Spring Festival) in mainland China. Please expect delayed responses, and we appreciate your patience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhance]: improve optimze()/merge()

3 participants