Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion .claude/skills/operating-weaviate-cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,23 @@ weaviate-cli delete collection --all --json

Key create options: `--multitenant`, `--auto_tenant_creation`, `--auto_tenant_activation`, `--shards N`, `--vectorizer <type>`, `--named_vector`, `--replication_deletion_strategy`, `--async_replication_config key=value` (repeatable; requires `--async_enabled` and Weaviate >= v1.36.0), `--object_ttl_type`, `--object_ttl_time`, `--object_ttl_filter_expired`, `--object_ttl_property_name` (only when `object_ttl_type=property`)

Mutable fields: `--async_enabled`, `--replication_factor`, `--vector_index`, `--description`, `--training_limit`, `--auto_tenant_creation`, `--auto_tenant_activation`, `--replication_deletion_strategy`, `--async_replication_config key=value` (repeatable), `--object_ttl_type`, `--object_ttl_time`, `--object_ttl_filter_expired`, `--object_ttl_property_name` (only when `object_ttl_type=property`)
Mutable fields: `--async_enabled`, `--replication_factor`, `--vector_index`, `--description`, `--training_limit`, `--auto_tenant_creation`, `--auto_tenant_activation`, `--replication_deletion_strategy`, `--async_replication_config key=value` (repeatable), `--object_ttl_type`, `--object_ttl_time`, `--object_ttl_filter_expired`, `--object_ttl_property_name` (only when `object_ttl_type=property`), `--drop_vector_index`

#### Drop a Named Vector Index

```bash
weaviate-cli update collection --collection Movies --drop_vector_index title_vector --json
```

**Destructive.** Deletes the index of one named vector from disk; the vector can no longer be
searched and the stored vectors are stripped by background cleanup. The vector **can be
re-created afterwards** (as a fresh, empty index via the client's `config.add_vector()`) once
the drop has finalized. Named vectors only, cannot be combined with `--vector_index`, and the
drop is applied asynchronously.

Requires Weaviate >= v1.39.0 started with `ENABLE_EXPERIMENTAL_ALTER_SCHEMA_DROP_VECTOR_INDEX_ENDPOINT=true`
-- the endpoint is experimental and disabled by default. Dropped vectors show as `none` in the
**Vector Index** column of `get collection` (e.g. `hnsw, none` for a partially dropped collection).

#### Async Replication Config

Expand Down
34 changes: 33 additions & 1 deletion .claude/skills/operating-weaviate-cli/references/collections.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,10 +104,42 @@ weaviate-cli update collection \
--json
```

Mutable fields: `--async_enabled`, `--replication_factor`, `--vector_index`, `--description`, `--training_limit`, `--auto_tenant_creation`, `--auto_tenant_activation`, `--replication_deletion_strategy`, `--async_replication_config`, `--object_ttl_type`, `--object_ttl_time`, `--object_ttl_filter_expired`, `--object_ttl_property_name` (only when `object_ttl_type=property`)
Mutable fields: `--async_enabled`, `--replication_factor`, `--vector_index`, `--description`, `--training_limit`, `--auto_tenant_creation`, `--auto_tenant_activation`, `--replication_deletion_strategy`, `--async_replication_config`, `--object_ttl_type`, `--object_ttl_time`, `--object_ttl_filter_expired`, `--object_ttl_property_name` (only when `object_ttl_type=property`), `--drop_vector_index`

**Immutable (cannot change after creation):** multitenant, vectorizer, named_vector, shards

## Drop a Named Vector Index (destructive)
```bash
weaviate-cli update collection --collection Movies --drop_vector_index title_vector --json
```

Removes the index of one named vector. The index is deleted from disk and the vector can no
longer be searched; the stored vectors are stripped by background cleanup. **The vector can be
re-created afterwards** as a fresh, empty index (e.g. via the client's `config.add_vector()`)
once the drop has finalized -- you then re-ingest to repopulate it.

- Named vectors only -- a collection with a single legacy vector is rejected.
- Cannot be combined with `--vector_index` (reconfiguring an index you are deleting is contradictory).
- Requires Weaviate **>= v1.39.0** started with `ENABLE_EXPERIMENTAL_ALTER_SCHEMA_DROP_VECTOR_INDEX_ENDPOINT=true`.
The endpoint is experimental and disabled by default; without the flag the server answers with a 500.
- The drop is applied **asynchronously**. A success message means Weaviate accepted the request,
not that the index is already gone. Poll `get collection --collection <name>` to observe completion.

Once dropped, the vector is reported by the server as `vectorIndexType: "none"` with no
`vectorIndexConfig`. In the collection listing it shows up as `none` in the **Vector Index**
column (e.g. `hnsw, none` when only some of the named vectors were dropped).

**Re-creation timing.** While the drop is in progress (vector shows `none`), re-creating the
same name is rejected. Once it finalizes (the vector disappears from the schema), the name can
be added again as a brand-new, empty index; the original index and any vectors already stripped
are not restored.

**Re-triggering a stalled drop.** Re-issuing `--drop_vector_index` on a vector that already
shows `none` is allowed and returns success. While cleanup is still running it is a no-op;
if the background cleanup had FAILED, it re-enqueues a fresh cleanup task. This is the only
way to recover a drop whose marker is stuck at `none`. The command reports that the vector
was already dropped and that it re-triggered cleanup.

**Async replication config examples (update):**
```bash
# Update async replication tuning on existing collection
Expand Down
6 changes: 6 additions & 0 deletions .claude/skills/operating-weaviate-cli/references/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ weaviate-cli create data \
- `--tenants "T1,T2"` -- Send data to specific tenants
- `--tenant_suffix` -- Prefix for auto-tenant names (default: "Tenant")

**Dropped vector indexes:** with `--randomize` on a named-vector collection, any vector whose
index was dropped (`vectorIndexType: "none"`, see `update collection --drop_vector_index`) is
**skipped automatically** -- generated objects do not carry it, so ingestion keeps working
instead of failing with "writes targeting it are rejected". A one-line note lists the skipped
vectors (suppressed under `--json`). Re-ingest the vector after re-creating its index.

## Update Data
```bash
weaviate-cli update data --collection "Movies" --limit 100 --randomize --json
Expand Down
7 changes: 6 additions & 1 deletion requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
weaviate-client>=4.21.0
# TEMPORARY: pinned to weaviate/weaviate-python-client#1991, which adds
# `collection.config.delete_vector_index()` and `VectorIndexType.NONE`, required by
# `update collection --drop_vector_index`. Swap back to a released `weaviate-client>=X`
# once the PR is merged and shipped. Use `@alter-schema-drop-vector-index` instead of the
# SHA to track new pushes to the PR.
weaviate-client @ git+https://github.com/weaviate/weaviate-python-client.git@2dbfaedf8d232057ce6d5a3b6a3479edcda62f94
click==8.1.7
twine
pytest
Expand Down
6 changes: 5 additions & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,11 @@ classifiers =
include_package_data = True
python_requires = >=3.9
install_requires =
weaviate-client>=4.21.0
# TEMPORARY: pinned to weaviate/weaviate-python-client#1991, which adds
# `collection.config.delete_vector_index()` and `VectorIndexType.NONE`, required by
# `update collection --drop_vector_index`. A direct URL cannot be published to PyPI,
# so this MUST go back to a released `weaviate-client>=X` before the next release.
weaviate-client @ git+https://github.com/weaviate/weaviate-python-client.git@2dbfaedf8d232057ce6d5a3b6a3479edcda62f94
click==8.1.7
semver>=3.0.2
numpy>=1.24.0
Expand Down
271 changes: 271 additions & 0 deletions test/unittests/test_managers/test_collection_manager.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import json
import pytest
from unittest.mock import MagicMock, patch, PropertyMock
from weaviate.exceptions import WeaviateConnectionError
Expand Down Expand Up @@ -1114,3 +1115,273 @@ def test_update_collection_async_replication_config_warns_on_old_version(
captured.out + captured.err
)
mock_collection.config.update.assert_called_once()


def _named_vector(index_type=None, vectorizer="none"):
"""Build a mock named vector config. `index_type=None` means the index was dropped."""
named_vector = MagicMock()
if index_type is None:
named_vector.vector_index_config = None
else:
named_vector.vector_index_config.vector_index_type.return_value = index_type
named_vector.vectorizer.vectorizer.value = vectorizer
return named_vector


def _named_vector_schema(vector_config):
"""Build a mock collection config that uses named vectors."""
return MagicMock(
vector_config=vector_config,
vectorizer=None,
vector_index_type=None,
replication_config=MagicMock(factor=1),
multi_tenancy_config=MagicMock(
enabled=False, auto_tenant_creation=False, auto_tenant_activation=False
),
)


def _drop_ready_collection(mock_client, vector_config=None):
"""Wire a mock collection whose named vector index can be dropped."""
mock_collections = MagicMock()
mock_client.collections = mock_collections
mock_client.collections.exists.side_effect = [True, True]
mock_client.get_meta.return_value = {"version": "1.39.0"}

mock_collection = MagicMock()
mock_client.collections.get.return_value = mock_collection
mock_collection.config.get.return_value = _named_vector_schema(
vector_config
if vector_config is not None
else {"title_vector": _named_vector("hnsw")}
)
return mock_collection


def test_update_collection_drop_vector_index(mock_client, mock_wvc_object_ttl):
"""--drop_vector_index deletes the index of the named vector, after the config update."""
mock_collection = _drop_ready_collection(mock_client)

manager = CollectionManager(mock_client)
manager.update_collection(
collection="TestCollection",
drop_vector_index="title_vector",
)

mock_collection.config.delete_vector_index.assert_called_once_with(
vector_name="title_vector"
)

# `config.update()` reads the whole schema and writes it back, so the drop has to
# come last -- otherwise a `vectorIndexType: "none"` vector is sent back to Weaviate.
call_names = [
call[0]
for call in mock_collection.config.mock_calls
if call[0] in ("update", "delete_vector_index")
]
assert call_names == ["update", "delete_vector_index"]


def test_update_collection_drop_vector_index_json_output(
mock_client, mock_wvc_object_ttl, capsys
):
"""The JSON payload reports the dropped vector and that the removal is async."""
_drop_ready_collection(mock_client)

manager = CollectionManager(mock_client)
manager.update_collection(
collection="TestCollection",
drop_vector_index="title_vector",
json_output=True,
)

payload = json.loads(capsys.readouterr().out)
assert payload["status"] == "success"
assert payload["dropped_vector_index"] == "title_vector"
assert "asynchronously" in payload["message"]


def test_update_collection_drop_vector_index_rejects_vector_index_combo(mock_client):
"""--drop_vector_index and --vector_index are mutually exclusive."""
mock_collections = MagicMock()
mock_client.collections = mock_collections

manager = CollectionManager(mock_client)
with pytest.raises(Exception) as exc_info:
manager.update_collection(
collection="TestCollection",
vector_index="hnsw",
drop_vector_index="title_vector",
)

assert "cannot be combined with --vector_index" in str(exc_info.value)
mock_collections.get.assert_not_called()


def test_update_collection_drop_vector_index_unknown_vector(
mock_client, mock_wvc_object_ttl
):
"""An unknown vector name fails before anything is updated or dropped."""
mock_collection = _drop_ready_collection(
mock_client,
vector_config={
"title_vector": _named_vector("hnsw"),
"body_vector": _named_vector("flat"),
},
)

manager = CollectionManager(mock_client)
with pytest.raises(Exception) as exc_info:
manager.update_collection(
collection="TestCollection",
drop_vector_index="missing_vector",
)

assert "Named vector 'missing_vector' does not exist" in str(exc_info.value)
assert "body_vector, title_vector" in str(exc_info.value)
mock_collection.config.update.assert_not_called()
mock_collection.config.delete_vector_index.assert_not_called()


def test_update_collection_drop_vector_index_without_named_vectors(
mock_client, mock_wvc_object_ttl
):
"""Only named vectors can be dropped, a legacy single-vector collection is rejected."""
mock_collection = _drop_ready_collection(mock_client, vector_config={})

manager = CollectionManager(mock_client)
with pytest.raises(Exception) as exc_info:
manager.update_collection(
collection="TestCollection",
drop_vector_index="title_vector",
)

assert "has no named vectors" in str(exc_info.value)
mock_collection.config.update.assert_not_called()
mock_collection.config.delete_vector_index.assert_not_called()


def test_update_collection_drop_vector_index_already_dropped_re_triggers(
mock_client, mock_wvc_object_ttl, capsys
):
"""Re-dropping an already-'none' vector is allowed: it re-triggers cleanup.

The server returns 200 for a repeat drop (no-op while cleanup runs, or a fresh
cleanup task if the previous one FAILED). The CLI must not block this -- it is the
only way for an operator to recover a stalled drop.
"""
mock_collection = _drop_ready_collection(
mock_client, vector_config={"title_vector": _named_vector(None)}
)

manager = CollectionManager(mock_client)
manager.update_collection(
collection="TestCollection",
drop_vector_index="title_vector",
)

mock_collection.config.delete_vector_index.assert_called_once_with(
vector_name="title_vector"
)
assert "already dropped" in capsys.readouterr().out


def test_update_collection_drop_vector_index_already_dropped_json_re_trigger(
mock_client, mock_wvc_object_ttl, capsys
):
"""JSON output still reports success and the re-trigger note for a repeat drop."""
_drop_ready_collection(
mock_client, vector_config={"title_vector": _named_vector(None)}
)

manager = CollectionManager(mock_client)
manager.update_collection(
collection="TestCollection",
drop_vector_index="title_vector",
json_output=True,
)

payload = json.loads(capsys.readouterr().out)
assert payload["status"] == "success"
assert payload["dropped_vector_index"] == "title_vector"
assert "re-trigger" in payload["message"]


def test_update_collection_drop_vector_index_warns_on_old_version(
mock_client, mock_wvc_object_ttl, capsys
):
"""Warn when --drop_vector_index is used against a server older than v1.39.0."""
mock_collection = _drop_ready_collection(mock_client)
mock_client.get_meta.return_value = {"version": "1.38.0"}

manager = CollectionManager(mock_client)
manager.update_collection(
collection="TestCollection",
drop_vector_index="title_vector",
)

captured = capsys.readouterr()
assert "Warning: --drop_vector_index requires Weaviate >= v1.39.0" in (
captured.out + captured.err
)
mock_collection.config.delete_vector_index.assert_called_once()


def test_update_collection_drop_vector_index_server_error_hints_at_env_var(
mock_client, mock_wvc_object_ttl
):
"""A server rejection points at the experimental feature flag it most likely needs."""
mock_collection = _drop_ready_collection(mock_client)
mock_collection.config.delete_vector_index.side_effect = Exception(
"endpoint is experimental and disabled by default"
)

manager = CollectionManager(mock_client)
with pytest.raises(Exception) as exc_info:
manager.update_collection(
collection="TestCollection",
drop_vector_index="title_vector",
)

assert "Failed to drop the index of named vector 'title_vector'" in str(
exc_info.value
)
assert "ENABLE_EXPERIMENTAL_ALTER_SCHEMA_DROP_VECTOR_INDEX_ENDPOINT=true" in str(
exc_info.value
)


def test_get_collection_lists_dropped_vector_index_as_none(mock_client, capsys):
"""A dropped index surfaces as `none` instead of crashing on a null index config."""
mock_client.collections = MagicMock()
mock_client.collections.list_all.return_value = ["Movies"]
mock_client.collections.get.return_value.config.get.return_value = (
_named_vector_schema({"title_vector": _named_vector(None)})
)

manager = CollectionManager(mock_client)
manager.get_collection(collection=None, json_output=True)

payload = json.loads(capsys.readouterr().out)
assert payload["collections"][0]["vector_index"] == "none"


def test_get_collection_lists_all_distinct_vector_index_types(mock_client, capsys):
"""Every distinct index type is listed, so a per-vector drop stays visible."""
mock_client.collections = MagicMock()
mock_client.collections.list_all.return_value = ["Movies"]
mock_client.collections.get.return_value.config.get.return_value = (
_named_vector_schema(
{
"title_vector": _named_vector("hnsw"),
"body_vector": _named_vector(None),
"extra_vector": _named_vector("hnsw"),
}
)
)

manager = CollectionManager(mock_client)
manager.get_collection(collection=None, json_output=True)

payload = json.loads(capsys.readouterr().out)
assert payload["collections"][0]["vector_index"] == "hnsw, none"
Loading
Loading