Add --drop_vector_index to update collection#171
Open
jfrancoa wants to merge 2 commits into
Open
Conversation
Adds `weaviate-cli update collection --drop_vector_index <name>` to drop the
index of a named vector (Weaviate 1.39 experimental endpoint, exposed by
weaviate-python-client#1991 as collection.config.delete_vector_index()).
- defaults.py: UpdateCollectionDefaults.drop_vector_index
- update.py: --drop_vector_index option, mutually exclusive with --vector_index
- collection_manager.py:
- drop runs AFTER config.update() (which reads-and-writes the whole schema),
so a "none"-marked vector is never PUT back to the server
- get collection listing renders a dropped vector as `none` and no longer
crashes on a null vector_index_config; shows all distinct index types
(e.g. `hnsw, none`)
- re-dropping an already-"none" vector is allowed (server returns 200): it is
a no-op while cleanup runs and re-enqueues a fresh task if the previous one
FAILED -- the only operator recovery path for a stalled drop
- version warning when the server is older than v1.39.0
- unit tests: happy path, validation, listing, and re-trigger
- requirements-dev.txt / setup.cfg: temporary pin to the client PR #1991 commit
(revert to a released weaviate-client before release)
- operating skill docs
Closes #170
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7 tasks
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Secrets | View in Orca |
A dropped vector index CAN be re-created (confirmed with the developer and verified live via config.add_vector() once the drop finalizes). Correct the docs/help/messages that wrongly said "cannot be re-created" / "irreversible": - update.py help, collection_manager success + conflict messages - operating skill (SKILL.md, collections.md): re-creation is allowed after the drop finalizes (fresh, empty index); "irreversible" removed create data: automatically skip named vectors whose index was dropped (vectorIndexType "none", reported as vector_index_config == None). The server rejects any object carrying a dropped vector, so every batch was failing after a drop. Now those vectors are excluded from generated objects (with an informational note, suppressed under --json) and ingestion keeps working. - data_manager.__ingest_data: filter dropped vectors, derive vectorizer from a live one, note skipped vectors - data.md: document the auto-skip - unit tests: skip one, skip all, skip none, --json suppression Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
weaviate-cli update collection --drop_vector_index <name>to drop the index of anamed vector — the Weaviate 1.39 experimental endpoint
(
DELETE /v1/schema/{Class}/vectors/{name}/index), exposed by the Python client inweaviate/weaviate-python-client#1991 as
collection.config.delete_vector_index(). The dropis destructive and applied asynchronously (a 200 means accepted, not done). The vector can
be re-created afterwards (a fresh, empty index via
config.add_vector()) once the dropfinalizes — verified live.
Closes #170
What changed
update.py/defaults.py—--drop_vector_index <name>, mutually exclusive with--vector_index; warns when the server is older than v1.39.0.collection_manager.pyconfig.update().config.update()reads the whole schema andPUTs it back, so dropping first would send a
vectorIndexType: "none"vector back to theserver. Ordering is asserted by a unit test.
get collectionlisting fix: a dropped vector'svector_index_configisNone; theold listing called
.vector_index_type()on it and crashed. It now rendersnoneandshows every distinct index type across named vectors (e.g.
hnsw, none).nonevector returns 200 on theserver — a no-op while cleanup runs, and a fresh cleanup task if the previous one FAILED
(the only operator recovery path for a stalled drop). The CLI reports the re-trigger
instead of blocking it.
create dataauto-skips dropped vectors — after a drop, the server rejects any objectcarrying the dropped vector, so
--randomizeingestion failed every batch. Generatedobjects now exclude any named vector whose index is dropped (
vectorIndexType: "none"),with an informational note (suppressed under
--json). Ingestion keeps working; re-ingestafter re-creating the index.
--vector_indexconflict, unknown/absent vector, listing(
none/hnsw, none), re-trigger, and thecreate dataskip (one / all / none /--json).336 pass.
create dataauto-skip).requirements-dev.txtandsetup.cfgpinweaviate-clientto the commit on thepython-client PR #1991 branch
(which adds
delete_vector_index()andVectorIndexType.NONE). That PR is not yet merged.Before a weaviate-cli release this must be reverted to a released
weaviate-client>=X. A gitURL cannot be published to PyPI, so this should not merge to a release tag as-is.
Test plan
make lintmake test(336 passed)stay 100% available during cleanup; dropped-vector search rejected immediately;
hnsw, noneshown mid-drop; re-drop of a
nonevector accepted;--vector_indexconflict rejected;vector re-created after finalize via
config.add_vector()and searchable again;create dataskips a dropped vector and ingests successfully (note shown;--jsonclean).🤖 Generated with Claude Code