Description
我在insert 20万的数据后通过子线程执行了optimize。optimize不会阻塞主进程的执行，但是在执行optimize的期间进行insert会卡主，一直到optimize执行完后才可继续插入数据
Steps to Reproduce
from concurrent.futures import ThreadPoolExecutor
import zvec
zvec.init(
log_type=zvec.LogType.CONSOLE,
query_threads=20,
optimize_threads=50,
)
collection = zvec.open(
path=str(DEMO_COLLECTION_DIR),
option=zvec.CollectionOption(read_only=False, enable_mmap=True)
)
self.executor = ThreadPoolExecutor(max_workers=4)
def heavy_task(self):
print("2.开始后台耗时任务")
self.collection.optimize()
print("3.耗时任务完成")
def insert(self,docs):
batch_size = 1000
st = time.time()
for idx in range(0,len(docs),batch_size):
print(f"1.loop {idx}")
docs_batch = docs[idx:idx+batch_size]
print("before insert")
st1 = time.time()
result = self.collection.insert(docs_batch)
print("after insert", time.time() - st1)
accumulated += len(docs_batch)
if accumulated >= 200000:
st2 = time.time()
print("before submit")
self.executor.submit(self.heavy_task)
print("after submit")
accumulated = 0
print(f" totil_time {time.time()-st} optimize_time {time.time()-st2}")
print(f"2.loop {idx}")Logs / Stack Trace
2.loop 198000
1.loop 199000
before insert
after insert 0.06690454483032227
before submit
2.开始后台耗时任务
after submit
totil_time 13.870309352874756 optimize_time 0.0009632110595703125
2.loop 199000
1.loop 200000
before insert
####### 在这里卡主，一直等线程内的optimize执行完才继续往下
3.耗时任务完成
after insert 44.16192555427551
2.loop 200000
1.loop 201000
Operating System
Ubuntu 20.04.2 LTS
Build & Runtime Environment
Python 3.10.12 zvec Version: 0.5.1 通过pip install zvec安装
Additional Context
Description
我在insert 20万的数据后通过子线程执行了optimize。optimize不会阻塞主进程的执行，但是在执行optimize的期间进行insert会卡主，一直到optimize执行完后才可继续插入数据
Steps to Reproduce
Logs / Stack Trace
2.loop 198000 1.loop 199000 before insert after insert 0.06690454483032227 before submit 2.开始后台耗时任务 after submit totil_time 13.870309352874756 optimize_time 0.0009632110595703125 2.loop 199000 1.loop 200000 before insert ####### 在这里卡主，一直等线程内的optimize执行完才继续往下 3.耗时任务完成 after insert 44.16192555427551 2.loop 200000 1.loop 201000Operating System
Ubuntu 20.04.2 LTS
Build & Runtime Environment
Python 3.10.12 zvec Version: 0.5.1 通过pip install zvec安装
Additional Context
git status— no uncommitted submodule changesCMAKE_BUILD_TYPE=DebugCOVERAGE=ON