Skip to content

Add --drop_vector_index to update collection#171

Open
jfrancoa wants to merge 2 commits into
mainfrom
jose/alter-schema-drop-index
Open

Add --drop_vector_index to update collection#171
jfrancoa wants to merge 2 commits into
mainfrom
jose/alter-schema-drop-index

Conversation

@jfrancoa

@jfrancoa jfrancoa commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds weaviate-cli update collection --drop_vector_index <name> to drop the index of a
named vector — the Weaviate 1.39 experimental endpoint
(DELETE /v1/schema/{Class}/vectors/{name}/index), exposed by the Python client in
weaviate/weaviate-python-client#1991 as collection.config.delete_vector_index(). The drop
is destructive and applied asynchronously (a 200 means accepted, not done). The vector can
be re-created afterwards (a fresh, empty index via config.add_vector()) once the drop
finalizes — verified live.

Closes #170

What changed

  • update.py / defaults.py--drop_vector_index <name>, mutually exclusive with
    --vector_index; warns when the server is older than v1.39.0.
  • collection_manager.py
    • The drop runs after config.update(). config.update() reads the whole schema and
      PUTs it back, so dropping first would send a vectorIndexType: "none" vector back to the
      server. Ordering is asserted by a unit test.
    • get collection listing fix: a dropped vector's vector_index_config is None; the
      old listing called .vector_index_type() on it and crashed. It now renders none and
      shows every distinct index type across named vectors (e.g. hnsw, none).
    • Re-drop is allowed: re-issuing the drop on an already-none vector returns 200 on the
      server — a no-op while cleanup runs, and a fresh cleanup task if the previous one FAILED
      (the only operator recovery path for a stalled drop). The CLI reports the re-trigger
      instead of blocking it.
  • create data auto-skips dropped vectors — after a drop, the server rejects any object
    carrying the dropped vector, so --randomize ingestion failed every batch. Generated
    objects now exclude any named vector whose index is dropped (vectorIndexType: "none"),
    with an informational note (suppressed under --json). Ingestion keeps working; re-ingest
    after re-creating the index.
  • Unit tests — drop happy path, --vector_index conflict, unknown/absent vector, listing
    (none / hnsw, none), re-trigger, and the create data skip (one / all / none / --json).
    336 pass.
  • Operating skill docs updated (drop semantics incl. re-creation; create data auto-skip).

⚠️ Temporary dependency pin (needs follow-up before release)

requirements-dev.txt and setup.cfg pin weaviate-client to the commit on the
python-client PR #1991 branch
(which adds delete_vector_index() and VectorIndexType.NONE). That PR is not yet merged.
Before a weaviate-cli release this must be reverted to a released weaviate-client>=X. A git
URL cannot be published to PyPI, so this should not merge to a release tag as-is.

Test plan

  • make lint
  • make test (336 passed)
  • Manual, live 3-node 1.39 cluster: drop accepted; sibling search / reads / clean writes
    stay 100% available during cleanup; dropped-vector search rejected immediately; hnsw, none
    shown mid-drop; re-drop of a none vector accepted; --vector_index conflict rejected;
    vector re-created after finalize via config.add_vector() and searchable again; create data skips a dropped vector and ingests successfully (note shown; --json clean).

🤖 Generated with Claude Code

Adds `weaviate-cli update collection --drop_vector_index <name>` to drop the
index of a named vector (Weaviate 1.39 experimental endpoint, exposed by
weaviate-python-client#1991 as collection.config.delete_vector_index()).

- defaults.py: UpdateCollectionDefaults.drop_vector_index
- update.py: --drop_vector_index option, mutually exclusive with --vector_index
- collection_manager.py:
  - drop runs AFTER config.update() (which reads-and-writes the whole schema),
    so a "none"-marked vector is never PUT back to the server
  - get collection listing renders a dropped vector as `none` and no longer
    crashes on a null vector_index_config; shows all distinct index types
    (e.g. `hnsw, none`)
  - re-dropping an already-"none" vector is allowed (server returns 200): it is
    a no-op while cleanup runs and re-enqueues a fresh task if the previous one
    FAILED -- the only operator recovery path for a stalled drop
- version warning when the server is older than v1.39.0
- unit tests: happy path, validation, listing, and re-trigger
- requirements-dev.txt / setup.cfg: temporary pin to the client PR #1991 commit
  (revert to a released weaviate-client before release)
- operating skill docs

Closes #170

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@orca-security-eu orca-security-eu Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

@jfrancoa
jfrancoa changed the base branch from master to main July 23, 2026 13:14
A dropped vector index CAN be re-created (confirmed with the developer and
verified live via config.add_vector() once the drop finalizes). Correct the
docs/help/messages that wrongly said "cannot be re-created" / "irreversible":

- update.py help, collection_manager success + conflict messages
- operating skill (SKILL.md, collections.md): re-creation is allowed after the
  drop finalizes (fresh, empty index); "irreversible" removed

create data: automatically skip named vectors whose index was dropped
(vectorIndexType "none", reported as vector_index_config == None). The server
rejects any object carrying a dropped vector, so every batch was failing after
a drop. Now those vectors are excluded from generated objects (with an
informational note, suppressed under --json) and ingestion keeps working.

- data_manager.__ingest_data: filter dropped vectors, derive vectorizer from a
  live one, note skipped vectors
- data.md: document the auto-skip
- unit tests: skip one, skip all, skip none, --json suppression

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@orca-security-eu orca-security-eu Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --drop_vector_index to weaviate-cli update collection

1 participant