docs: Weaviate 1.39 release#479
Open
g-despot wants to merge 8 commits into
Open
Conversation
Remove the schema_tx metrics, which were deleted from core in 1.39. Promote the Boost API note from preview to generally available, and rename the now-misleading BoostPreview import alias to BoostNote across the five pages that render it.
In v1.39 the HNSW commit log compactor owns the full on-disk lifecycle of the vector index. Snapshots are always created and managed automatically, and the five PERSISTENCE_HNSW_SNAPSHOT_* environment variables are inert no-ops that log a startup warning when set. Docs serve the latest three supported minors, so the pre-v1.39 configuration guidance is retained and scoped to v1.31 through v1.38 rather than removed. - configuration/hnsw-snapshots.md: lead with the v1.39 behavior and the list of ignored variables; move the existing configuration reference into a section scoped to earlier versions. - concepts/storage.md: describe snapshots as part of the storage engine rather than an opt-in speedup, cover the streaming on-disk merge and the crash-safety mechanisms, and scope the previous conditional creation and snapshot fallback behavior to earlier versions. - env-vars/index.md: mark the five variables deprecated in v1.39 while preserving what they did for readers on earlier versions. - feature-notes/hnsw-snapshots.mdx: note the v1.39 change on both pages that consume the include.
The v1.39 crash-safety paragraph said a compacted segment that cannot be read in full is dropped in favor of the snapshot and the remaining clean segments. That is not what the loader does. For both raw commit logs and compacted .sorted/.condensed segments it calls truncateToLastValidRecord, keeping every record written before the corruption and leaving the file valid for later compaction. Only the damaged tail is lost. The two outcomes differ in recall, so describe the retention explicitly and phrase both cases as the single mechanism they actually are. Note that the struct comment at compact/loader.go:73-74 and the matching one at hnsw/startup.go:101-102 still describe the discard behavior and contradict the implementation. Tracked separately on the core side.
The card read "Configure HNSW index snapshots for faster recovery and
startup." There is nothing to configure in v1.39, but the page still
serves v1.37 and v1.38 readers who can configure it, so the description
now leads with the outcome instead of the action and reads correctly for
both audiences.
Keeps the imperative benefit-first voice of the neighbouring cards
("Reduce memory usage with...", "Enable the built-in..."), and drops the
restatement of the card title, which the title already supplies.
Blocking items: - Move the v1.39 startup-failure behavior out of the historical note and into the main durability prose. It was the last sentence of a box titled "Behavior before v1.39", which told v1.39 readers to skip the highest-consequence fact in the release. - Split the durability paragraph in three so commit logs and snapshots are not conflated. Commit logs, including compacted ones, self-heal by truncation; snapshots are never truncated or repaired and an unreadable one fails the index load. - Stop calling HNSW compaction output a "segment", which contradicts "the vector index does not use segmentation" earlier on the same page. Vocabulary is now commit log, compacted commit log, snapshot, compactor. - Replace en dashes in the env-var rows with "through", matching the other two files. Also: - Compound title on the feature-note include, per the rq-8bit precedent. - Halve the repeated boilerplate in the five env-var rows and point the historical half at a new stable anchor on the pre-v1.39 section, so readers on earlier versions land on the part that applies to them. - Drop "periodically" from the snapshot lead-in (compaction is driven by the compactor, not a timer) and stop framing a snapshot as conditional. - Retitle both scoping notes to name v1.31 through v1.38 explicitly, since snapshots did not exist before v1.31. - Add a caution to the configuration page covering the fail-to-start behavior, which every deprecated env-var row links to. - Name the configuration file that silently accepts these settings. The recovery advice was verified against the compactor deleting its input files after a successful snapshot write, and the loader filtering covered commit logs, which together confirm no local copy survives.
![orca-security-eu[bot]](https://avatars.githubusercontent.com/in/276244?s=60&v=4){kind=small}
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
 Passed |
Infrastructure as Code |  0  0  0  0 |
View in Orca |
| Passed |
SAST | 0 0 0 0 |
View in Orca |
| Passed |
Secrets | 0 0 0 0 |
View in Orca |
| Passed |
Vulnerabilities | 0 0 0 0 |
View in Orca |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release PR aggregating the ready 1.39 documentation into
main. Assembled on thev1-39/mainintegration branch from the sub-PRs below.Included
#474 reference updates (
v1-39/reference-updates)schema_tx_*Prometheus metrics deleted from core in 1.39 (monitoring.md).BoostPreviewinclude alias toBoostNoteacross the five search pages that render it.#475 HNSW snapshots are automatic in 1.39 (
v1-39/hnsw-snapshots-deprecation)PERSISTENCE_HNSW_SNAPSHOT_*env vars are now deprecated no-ops (env-vars/index.md).hnsw-snapshots.mdsplit into a 1.39 section and a pre-1.39 section (the docs serve the latest three minors);concepts/storage.mdrewritten; config index card reworded.Each component passed a verifier build and an editor pass individually, and the two touch disjoint files (clean merge). This PR's CI is the authoritative combined build.
Deliberately NOT included
ChunkSizefix, backup chunking vars,text2vec-googlelocation) targetsmaindirectly as its own PR, since it fixes docs for current releases and is not 1.39-gated.Flag for the release owner
#475 documents that in 1.39 an unreadable HNSW snapshot fails index startup, where 1.38 self-healed. That behavior change is tracked as a pre-GA engineering question (does the fallback get restored?). The env-var deprecation content is solid regardless; only that one paragraph in
concepts/storage.mdis contingent. If engineering restores the fallback before GA, that paragraph must be reverted.