Skip to content

[Bug]: Issue with creating indexes and executing optimize for certain data types when using cosine metric scenario #78

@liututu12

Description

@liututu12

Description

Issue Description

Scenario 1:
For the "vector_int8_field" vector data type, create three types of indexes: IVF, Flat, and Hnsw, all using cosine similarity metric. Creating indexes fails in all cases. Specific script assertion error messages are as follows:
Script execution error message:
AssertionError: result=[{"code":8, "message":"Create vector column indexer failed: vector_int8_field"}.....}];

Index parameters are as follows:
HnswIndexParam(metric_type=MetricType.COSINE, m=24, ef_construction=150, );
FlatIndexParam(metric_type=MetricType.COSINE, );
IVFIndexParam(metric_type=MetricType.COSINE, n_list=150, n_iters=15, use_soar=False, );

Scenario 2:
For all supported vector data types, when creating ivf index type and selecting cosine metric, calling the optimize method results in an error. Error message is as follows:
../../../.conda/lib/python3.9/site-packages/zvec/model/collection.py:154: in optimize
self._obj.Optimize(option)
E RuntimeError: Failed to merge index

Steps to Reproduce

# Reproduction Steps
Scenario 1:
1. Download test/test_recall branch code,
2. Execute pytest test_collection_recall.py script;
3. Pay attention to the following 3 test cases that fail in the execution results:
test_recall_with_single_vector_valid[1-10-2000-full_schema_new4], test_recall_with_single_vector_valid[1-10-2000-full_schema_new7], test_recall_with_single_vector_valid[1-10-2000-full_schema_new11] - 3 test cases fail:
AssertionError: result=[{"code":8, "message":"Create vector column indexer failed: vector_int8_field"}, {"code":8, "message":"vector indexer not found for field: vector_int8_field"},...

Scenario 2:
1. Download test/test_recall branch code;
2. Comment out line 73 in support_helper.py: #DataType.VECTOR_INT8: "vector_int8_field", Reason for commenting: To temporarily bypass Scenario 1 errors, otherwise Scenario 2 error messages cannot be exposed.
3. Execute pytest test_collection_recall.py script; pay attention to the test result where
test_recall_with_single_vector_valid[1-10-2000-full_schema_new11] directly fails: RuntimeError: Failed to merge index

Logs / Stack Trace

Operating System

Linux Docker environment

Build & Runtime Environment

NA

Additional Context

  • I've checked git status — no uncommitted submodule changes
  • I built with CMAKE_BUILD_TYPE=Debug
  • This occurs with or without COVERAGE=ON
  • The issue involves Python ↔ C++ integration (pybind11)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status

In progress

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions