From 45dfe42c78cf6a1823a39379a69c715a08a38328 Mon Sep 17 00:00:00 2001 From: robfrank Date: Fri, 10 Jul 2026 12:24:50 +0200 Subject: [PATCH 1/5] docs(26.7.2): document HA durable Raft storage and Bolt temporal breaking changes Cover the two 26.7.2 breaking changes and related HA contracts: - arcadedb.ha.raftPersistStorage now defaults to true (durable Raft storage). Correct the default in settings.adoc and the HA settings table, add ha.raftStorageDirectory to the HA table, and add a "Durable Raft Storage" section. Fix the now-stale "wiped on every restart / do not copy" guidance in upgrade.adoc (What to copy, HA cluster upgrade, Docker/Kubernetes). - Bolt temporal values (date/time/datetime/...) are now native PackStream structures instead of ISO-8601 strings. Add a breaking- change NOTE to the Bolt data type mapping and a bolt-datatypes anchor. - Add the committed-remotely response contract (TransactionCommittedRemotelyException -> HTTP 409, do not retry). - Note leader auto-recovery of a follower's replication channel after a pod-IP/DNS change. - Add 26.7.2 entries to the upgrade Breaking changes section. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../asciidoc/how-to/connectivity/bolt.adoc | 6 ++++ src/main/asciidoc/how-to/operations/ha.adoc | 34 +++++++++++++++++-- .../asciidoc/how-to/operations/upgrade.adoc | 31 +++++++++++++---- src/main/asciidoc/reference/settings.adoc | 4 +-- 4 files changed, 65 insertions(+), 10 deletions(-) diff --git a/src/main/asciidoc/how-to/connectivity/bolt.adoc b/src/main/asciidoc/how-to/connectivity/bolt.adoc index 67d3e9cd..d081fcf5 100644 --- a/src/main/asciidoc/how-to/connectivity/bolt.adoc +++ b/src/main/asciidoc/how-to/connectivity/bolt.adoc @@ -420,6 +420,7 @@ try (Session read = driver.session(SessionConfig.builder() } ---- +[[bolt-datatypes]] ===== Data Type Mapping ArcadeDB emits native BOLT/PackStream structures for the following, so drivers receive typed objects (not strings): @@ -434,6 +435,11 @@ ArcadeDB emits native BOLT/PackStream structures for the following, so drivers r Values without a native BOLT type — `RID` and `UUID` — are serialized as strings. `BigDecimal` and out-of-range `BigInteger` values are converted to `double` and may lose precision. +[NOTE] +==== +*Changed in v26.7.2 (breaking for Bolt clients).* Temporal values (`Date`, `Time`, `LocalTime`, `DateTime`, `LocalDateTime`) are now carried as native PackStream temporal structures in *both* directions. Before v26.7.2 they were sent as ISO-8601 strings, and inbound datetime query parameters were silently dropped. A client that previously read a temporal property as a `String` must now read the native temporal type — e.g. `Value.asZonedDateTime()` or `Value.asLocalDate()` — exactly as it would against Neo4j. See <>. +==== + ===== Error Codes Server errors are mapped to Neo4j-style structured status codes so driver-side error handling and retry logic behave as expected: diff --git a/src/main/asciidoc/how-to/operations/ha.adoc b/src/main/asciidoc/how-to/operations/ha.adoc index 6535ab41..c273bcd7 100644 --- a/src/main/asciidoc/how-to/operations/ha.adoc +++ b/src/main/asciidoc/how-to/operations/ha.adoc @@ -317,6 +317,32 @@ Common causes of leader unavailability include: When a replica rejoins after being offline, it catches up via the Raft log automatically; if it has fallen behind past the log purge boundary, the leader streams a full database snapshot over HTTP and the replica installs it atomically. +Since v26.7.2, the leader also *auto-recovers a follower's replication channel* that is wedged on a stale peer address — for example after a Kubernetes pod is rescheduled onto a new IP. Previously such a follower stayed silently disconnected and the cluster ran at bare quorum until an operator forced a leadership transfer; the leader now re-resolves the peer and re-establishes the channel on its own. + +[[ha-durable-raft-storage]] +===== Durable Raft Storage (from v26.7.2) + +[IMPORTANT] +==== +Since v26.7.2, `arcadedb.ha.raftPersistStorage` defaults to `true` (it was `false` before). The per-node Raft log under `raft-storage-` is now *persisted across restarts*, and `arcadedb.ha.raftStorageDirectory` must live on *durable* storage (a persistent volume, not an ephemeral or `tmpfs` mount). +==== + +With ephemeral Raft storage, wiping the log on restart could leave the cluster in a broken state: + +* a lagging follower that came back after a *full-cluster cold restart* could permanently diverge and fail to rejoin, surfacing as a `WALVersionGapException`; or +* the cluster could silently re-form as a fresh single-node cluster, dropping committed history. + +Durable storage lets a restarted node rejoin by *replaying its own persisted log* rather than always requiring a full snapshot resync from the leader. + +*Migration:* mount `raft-storage-*` (or the directory named by `arcadedb.ha.raftStorageDirectory`) on durable storage before upgrading. On Kubernetes, back it with a `PersistentVolumeClaim` — see <>. A throwaway or test cluster that intentionally wants ephemeral behavior can opt out with `arcadedb.ha.raftPersistStorage=false`. + +[[ha-remote-commit-contract]] +===== Committed-remotely response contract (from v26.7.2) + +When a write is forwarded from a replica to the leader (or issued directly to the leader), it is possible for the transaction to be *committed cluster-wide by quorum* and then fail to apply on the node that is answering the client. This is a distinct outcome from an ordinary failure: the data is durably committed, so retrying would create a duplicate. + +Since v26.7.2 this case has its own contract. The server raises a `TransactionCommittedRemotelyException`, mapped over HTTP to *status `409 Conflict`*. A client that receives it must *not* retry the transaction — the write already succeeded cluster-wide; it should instead re-read to observe the committed state. This is different from a transient conflict (a genuine write-write conflict or `DeadlockException`), which remains safe to retry. + [[ha-offline-bootstrap]] ===== Offline Cluster Bootstrap (from v26.5.1) @@ -546,8 +572,12 @@ A complete list of all HA parameters is available in <> +|true + +|`+arcadedb.ha.raftStorageDirectory+` +|Parent directory for the per-node `raft-storage-` sub-folder. Empty (default) uses the server root path. With durable storage enabled by default, point this at a persistent volume. See <> +|(empty) |`+arcadedb.ha.stopServerOnReplicationFailure+` |If true, the JVM exits after exhausting step-down retries on a phase-2 replication failure. Default false keeps the server up and logs CRITICAL diff --git a/src/main/asciidoc/how-to/operations/upgrade.adoc b/src/main/asciidoc/how-to/operations/upgrade.adoc index 2c5ce497..dbd8eb68 100644 --- a/src/main/asciidoc/how-to/operations/upgrade.adoc +++ b/src/main/asciidoc/how-to/operations/upgrade.adoc @@ -9,6 +9,24 @@ The migration is transparent and happens lazily on first open, so the only opera This section lists behavior changes that may require action when upgrading. +[discrete] +====== 26.7.2 — Durable Raft storage is now the default + +`arcadedb.ha.raftPersistStorage` now defaults to `true` (previously `false`). The per-node Raft log under `raft-storage-` is persisted across restarts so a node can rejoin by replaying its own log instead of always requiring a full snapshot resync. Wiping the log on restart could previously diverge a lagging follower after a full-cluster cold restart (`WALVersionGapException`) or silently re-form a fresh single-node cluster. + +*Impact:* HA clusters only. + +*Action:* ensure `arcadedb.ha.raftStorageDirectory` (or the default `raft-storage-*` location under the server root) lives on *durable* storage — a persistent volume, not an ephemeral or `tmpfs` mount. On Kubernetes, back it with a `PersistentVolumeClaim`. A throwaway/test cluster can opt out with `arcadedb.ha.raftPersistStorage=false`. See <>. + +[discrete] +====== 26.7.2 — Bolt temporal values are now native PackStream types + +Over the Neo4j Bolt protocol, `date`, `time`, `localtime`, `datetime` and `localdatetime` are now carried as *native PackStream temporal structures* in both directions, matching Neo4j. Previously they were serialized as ISO-8601 strings (and inbound datetime query parameters were silently dropped). + +*Impact:* Bolt/Neo4j-driver clients that read a temporal property as a `String`. + +*Action:* read temporal properties with the driver's native temporal accessors (e.g. `Value.asZonedDateTime()`, `asLocalDate()`), exactly as you would against Neo4j. See <>. + [discrete] ====== 26.7.1 — Cypher `IS TYPED INT` is now 32-bit (`INTEGER`) @@ -55,8 +73,8 @@ Only a few of them hold user-owned state; the rest are recreated from the releas |Server log files. Safe to start with an empty `log/` directory; the release ships an empty one. Copy only if you want to keep historical logs alongside the new install. |`raft-storage-/` (HA only) -|Skip -|Per-node Raft log segments, created at runtime when HA is enabled. By default this directory is recreated on every restart, so there is nothing to migrate. See <> for details. +|*Copy* (since v26.7.2) +|Per-node Raft log segments, created at runtime when HA is enabled. Since v26.7.2 this storage is *durable by default* (`arcadedb.ha.raftPersistStorage=true`) and must be preserved so a node can rejoin by replaying its log instead of a full snapshot resync. On releases before v26.7.2, or when `raftPersistStorage` is explicitly `false`, it is ephemeral and can be skipped. See <> for details. |`replication/` |Skip @@ -110,9 +128,10 @@ What to migrate on each node: * *`databases/`* — copy, exactly as in the single-server case. * *`config/`* — copy the same files as in the single-server case. The cluster's identity is derived from `arcadedb.ha.clusterName` and the `root` password (the inter-node token is computed from them at startup), so as long as every node keeps the same cluster name and the same root password, the upgraded cluster reattaches to itself without any extra step. -* *`raft-storage-/`* — *do not copy*. -This directory holds the local Raft log and is wiped and re-created on every restart. -After the upgraded node rejoins the cluster, the leader replays missing entries via Raft or, if the node has fallen behind the purge boundary, streams a full snapshot over HTTP automatically. +* *`raft-storage-/`* — *copy* (since v26.7.2). +Since v26.7.2 the local Raft log is durable by default (`arcadedb.ha.raftPersistStorage=true`), so preserve this directory across the upgrade to let the node rejoin by replaying its own log. +Whether or not it is preserved, after the upgraded node rejoins the cluster the leader replays any missing entries via Raft or, if the node has fallen behind the purge boundary, streams a full snapshot over HTTP automatically. +On releases before v26.7.2 (or with `raftPersistStorage=false`) this directory is ephemeral and can be skipped. * *`backups/` and `log/`* — same as single-server. Recommended rolling upgrade flow (zero downtime, requires quorum at all times): @@ -134,7 +153,7 @@ The only data that survives across container restarts is what lives on a persist * Mount `/home/arcadedb/databases` on a persistent volume — this is the equivalent of "copy the `databases/` folder". * Mount `/home/arcadedb/config` only if you have edited the files listed in <>; otherwise let the image ship its defaults. -* Do *not* mount `/home/arcadedb/raft-storage-*` on a persistent volume; the Raft log is ephemeral and is rebuilt from peers on every pod restart. +* Mount `/home/arcadedb/raft-storage-*` (or the path set by `arcadedb.ha.raftStorageDirectory`) on a persistent volume. Since v26.7.2 the Raft log is durable by default (`arcadedb.ha.raftPersistStorage=true`) so a restarted pod can replay its own log instead of resyncing from peers. On releases before v26.7.2, or with `raftPersistStorage=false`, the log is ephemeral and this mount is unnecessary. * Mount `/home/arcadedb/backups` if you want to retain backup archives across pod restarts. For Kubernetes specifics (StatefulSet, headless service, init container for pre-staging a database), see <>. diff --git a/src/main/asciidoc/reference/settings.adoc b/src/main/asciidoc/reference/settings.adoc index c8aac131..78c6601b 100644 --- a/src/main/asciidoc/reference/settings.adoc +++ b/src/main/asciidoc/reference/settings.adoc @@ -150,8 +150,8 @@ The following plugins are available for ArcadeDB Server: |`ha.quorum`|Write quorum: `majority` or `all`. _Legacy values `none`, `one`, `two`, `three` removed in v26.4.1_|String|majority |`ha.quorumTimeout`|Timeout in ms waiting for the quorum acknowledgment|Long|10000 |`ha.raftPort`|TCP/IP port for Raft gRPC communication. Used as the default when HA_SERVER_LIST entries do not specify an explicit port. _Since v26.4.1_|Integer|2434 -|`ha.raftPersistStorage`|If true, the Raft storage directory is preserved across restarts, enabling node rejoin without a full snapshot resync. _Since v26.4.1_|Boolean|false -|`ha.raftStorageDirectory`|Parent directory under which the per-node Raft storage sub-folders (`raft-storage-`) are created. When empty (the default), the server root path (`arcadedb.server.rootPath`) is used, preserving the existing behaviour. Set this to a writable, statically-named path (e.g. a PVC mount) on Kubernetes StatefulSets with `readOnlyRootFilesystem: true`, where the dynamically-named `raft-storage-*` directory under the read-only server root cannot be created or mounted. _(Available since v26.6.1)_|String| +|`ha.raftPersistStorage`|If true, the Raft storage directory is preserved across restarts, enabling node rejoin by replaying its persisted log instead of a full snapshot resync. *Defaults to `true` since v26.7.2* (previously `false`): wiping the Raft log on restart could permanently diverge a lagging follower after a full-cluster cold restart (`WALVersionGapException`) or silently re-form a fresh single-node cluster. Ensure `ha.raftStorageDirectory` lives on durable storage. A throwaway/test cluster can opt out with `false`. _Since v26.4.1_|Boolean|true +|`ha.raftStorageDirectory`|Parent directory under which the per-node Raft storage sub-folders (`raft-storage-`) are created. When empty (the default), the server root path (`arcadedb.server.rootPath`) is used. Because `ha.raftPersistStorage` now defaults to `true`, this directory must live on *durable* storage (a persistent volume, not an ephemeral/`tmpfs` mount) so a restarted node can replay its log and rejoin without a full snapshot resync. Set this to a writable, statically-named path (e.g. a PVC mount) on Kubernetes StatefulSets with `readOnlyRootFilesystem: true`, where the dynamically-named `raft-storage-*` directory under the read-only server root cannot be created or mounted. _(Available since v26.6.1)_|String| |`ha.ratisRestartMaxRetries`|Maximum consecutive Ratis restart attempts by the health monitor before the server shuts down for cluster-level recovery. _Since v26.4.1_|Integer|10 |`ha.readConsistency`|Default read consistency for replica reads: `eventual`, `read_your_writes`, `linearizable`. _Since v26.4.1_|String|read_your_writes |`ha.replicationChunkMaxSize`|Maximum channel chunk size for replicating messages between servers|Integer|16777216 From 6bd4a9be3dc468d2c40fcbed9ff8dc6898ea9a61 Mon Sep 17 00:00:00 2001 From: robfrank Date: Fri, 10 Jul 2026 12:26:49 +0200 Subject: [PATCH 2/5] docs(26.7.2): document security-hardening changes to privileges, functions and triggers Cover the 26.7.2 security advisories that change user-visible behavior: - Add the updateDatabaseSettings group permission (GHSA-8vr5-263f-x5r3) to groups.adoc, including the auto-migration of admin groups and the new default admin access list. - Document required privileges for DEFINE FUNCTION in sql-custom- functions.adoc: LANGUAGE js now requires updateSecurity, sql/cypher require updateSchema (GHSA-vwjc-v7x7-cm6g), plus the polyglot sandbox (IOAccess.NONE / PolyglotAccess.NONE). - Expand the JS trigger sandbox note in sql-triggers.adoc with the narrowed host-class allow-list (java.util/java.time/java.math only; java.lang.* removed) and the RCE advisory GHSA-x9f9-r4m8-9xc2. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../asciidoc/how-to/operations/groups.adoc | 10 +++++++--- .../reference/sql/sql-custom-functions.adoc | 19 +++++++++++++++++++ .../asciidoc/reference/sql/sql-triggers.adoc | 2 +- 3 files changed, 27 insertions(+), 4 deletions(-) diff --git a/src/main/asciidoc/how-to/operations/groups.adoc b/src/main/asciidoc/how-to/operations/groups.adoc index e29b1664..385c0d99 100644 --- a/src/main/asciidoc/how-to/operations/groups.adoc +++ b/src/main/asciidoc/how-to/operations/groups.adoc @@ -21,8 +21,9 @@ Where: The wildcard "`*`" represents all the types that are not defined in this configuration. * `access` is the array containing the allowed permissions for the group. The supported permission at group level are: -** `updateSecurity`: to update the security settings (create, modify and delete users, groups, etc.) +** `updateSecurity`: to update the security settings (create, modify and delete users, groups, etc.). Since v26.7.2 this also gates defining a JavaScript function via `DEFINE FUNCTION ... LANGUAGE js` — see <>. ** `updateSchema`: to update the database schema (create, modify and drop buckets, types and indexes) +** `updateDatabaseSettings` (since v26.7.2): to change per-database settings — e.g. `ALTER DATABASE`, `ALTER TYPE ... BUCKETSELECTIONSTRATEGY`, materialized-view / continuous-aggregate refresh and downsampling policies. Previously several of these mutators were reachable without a dedicated privilege. * `readTimeout` if present, specify the maximum timeout for read operations. -1 means no limits. If set, all the read operations (lookups and queries) will be limited to maximum `` milliseconds. This is useful to limit users to execute expensive commands and queries impacting the performance of the server and therefore other connected users. @@ -71,7 +72,8 @@ The default settings for the `admin` group are: { "access": [ "updateSecurity", - "updateSchema" + "updateSchema", + "updateDatabaseSettings" ], "resultSetLimit": -1, "readTimeout": -1, @@ -88,7 +90,9 @@ The default settings for the `admin` group are: } ``` -Which allows to execute any operation against the security, the schema and records. +Which allows to execute any operation against the security, the schema, the database settings and records. + +NOTE: The `updateDatabaseSettings` permission was added in v26.7.2. When ArcadeDB opens an older `server-groups.json`, it automatically migrates existing `admin` groups to include it, so upgraded clusters keep full administrative access without a manual edit. Here is an example for an append-only group: diff --git a/src/main/asciidoc/reference/sql/sql-custom-functions.adoc b/src/main/asciidoc/reference/sql/sql-custom-functions.adoc index 42cca23a..0a1c2a40 100644 --- a/src/main/asciidoc/reference/sql/sql-custom-functions.adoc +++ b/src/main/asciidoc/reference/sql/sql-custom-functions.adoc @@ -59,6 +59,18 @@ DEFINE FUNCTION . "" [PARAMETERS [,*]] LANGUAGE Functions defined with `DEFINE FUNCTION` (SQL, JavaScript and OpenCypher) are stored with the database schema: they survive a restart and, in a High Availability cluster, are automatically replicated to all the servers. `DELETE FUNCTION` removes them permanently. Java functions are the exception, they are registered programmatically at startup (see <> below). ==== +[IMPORTANT] +==== +**Required privileges (since v26.7.2)** + +Defining a function over a remote connection is gated by the caller's group permissions (see <>): + +* `DEFINE FUNCTION ... LANGUAGE sql` and `LANGUAGE opencypher` require the `updateSchema` permission, like any other schema change. +* `DEFINE FUNCTION ... LANGUAGE js` requires the stronger `updateSecurity` permission, because a JavaScript body runs host code. Before v26.7.2 the SQL route to JavaScript bypassed this gate, letting a user with only schema access define and run arbitrary JavaScript. + +Embedded databases, schema loading and HA replication are unaffected — the guard only applies to remote commands. +==== + ====== SQL Language Functions SQL functions execute SQL queries and return the result of the first row's projection. @@ -86,6 +98,13 @@ SELECT `utils.currentYear`(); JavaScript functions use GraalVM's JavaScript engine for complex calculations and data transformations. +[NOTE] +==== +**Sandbox (since v26.7.2)** + +The polyglot engine that runs a `LANGUAGE js` function is locked down: it executes with host file/URL access disabled (`IOAccess.NONE`, so `load(path|url)` cannot read local files or reach network resources) and cross-language access disabled (`PolyglotAccess.NONE`, so it cannot pivot into another guest language). Combined with the `updateSecurity` requirement above, this confines JavaScript functions to pure in-process computation over their arguments. +==== + *Examples* [source,sql] diff --git a/src/main/asciidoc/reference/sql/sql-triggers.adoc b/src/main/asciidoc/reference/sql/sql-triggers.adoc index 55a9f964..805da648 100644 --- a/src/main/asciidoc/reference/sql/sql-triggers.adoc +++ b/src/main/asciidoc/reference/sql/sql-triggers.adoc @@ -652,7 +652,7 @@ DROP TRIGGER user_audit; 5. *Transaction Context*: Triggers execute within the same transaction as the triggering operation. If a trigger fails, the entire transaction rolls back. -6. *JavaScript Sandboxing*: JavaScript triggers run in a sandboxed environment with limited access to Java packages for security. +6. *JavaScript Sandboxing*: JavaScript triggers run in a sandboxed GraalVM environment with a restricted host-class allow-list. Since v26.7.2 the allow-list is limited to the benign value packages `java.util.*`, `java.time.*` and `java.math.*`; `java.lang.*` was removed so a trigger can no longer look up `java.lang.Runtime` and reach `Runtime.getRuntime().exec(...)` for OS command execution (advisory `GHSA-x9f9-r4m8-9xc2`). JavaScript's native `String`, `Math` and `Number` cover most trigger logic, so dropping `java.lang.*` rarely affects real triggers. ===== See Also From 89dadf0f1a5d399e2205799151ce1618f2b04528 Mon Sep 17 00:00:00 2001 From: robfrank Date: Fri, 10 Jul 2026 12:29:33 +0200 Subject: [PATCH 3/5] docs(#5143): document configurable posting-weight quantization for LSM_SPARSE_VECTOR The sparse vector index now exposes posting-weight quantization through the weightQuantization index METADATA key (INT8 default, FP16, FP32), at parity with the dense LSM_VECTOR quantization knob. Previously the choice was hard-wired to INT8. - sql-indexes.adoc: add weightQuantization to the LSM_SPARSE_VECTOR METADATA example and a values/trade-off table. - vector-search.adoc: remove weight quantization from the "tracked separately" future-work list and point to CREATE INDEX; fix a pre-existing typo flagged by the typos hook. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/main/asciidoc/concepts/vector-search.adoc | 6 ++++-- .../asciidoc/reference/sql/sql-indexes.adoc | 20 ++++++++++++++++++- 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/src/main/asciidoc/concepts/vector-search.adoc b/src/main/asciidoc/concepts/vector-search.adoc index 481aa3a2..50d36ff0 100644 --- a/src/main/asciidoc/concepts/vector-search.adoc +++ b/src/main/asciidoc/concepts/vector-search.adoc @@ -137,7 +137,7 @@ CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA { }; ---- -The factory rejects two misconfigurations at index-creation time so the failure surfaces immediately rather than as silent mis-scaling later: +The factory rejects two misconfigurations at index-creation time so the failure surfaces immediately rather than as silent incorrect scaling later: * `encoding=INT8` with a non-`BINARY` property (or `encoding=FLOAT32` with a `BINARY` property) -- the property type and encoding must agree. * `encoding=INT8` combined with `quantization=INT8` -- redundant: the property is already byte-quantized at the wire level, so the index-internal scalar quantizer would re-quantize the dequantized floats. Pick one (`encoding=INT8` for payload/storage savings, `quantization=INT8` for index-internal compression), not both. @@ -505,7 +505,9 @@ Options map (5th argument) supports `filter` (allowed-RIDs whitelist), `groupBy` Retrieval uses document-at-a-time WAND with per-dim `max_weight` upper bounds: per-dim cursors traverse postings in RID order, the algorithm pivots to the smallest RID whose cumulative upper bound can still beat the current K-th best score, and skips cursors below the pivot via direct seeks. Only dot-product similarity is supported (cosine on sparse vectors is conventionally handled by L2-normalizing both the query and stored vectors at insert time). -The MVP works well up to roughly 10M sparse vectors. The scaling story past that point (BlockMax-WAND with per-page bounds, weight quantization, parallel per-segment scoring) is tracked separately. +The MVP works well up to roughly 10M sparse vectors. The scaling story past that point (BlockMax-WAND with per-page bounds, parallel per-segment scoring) is tracked separately. + +Posting-weight quantization is configurable since v26.7.2 via the `weightQuantization` index `METADATA` key (`INT8` — the default — `FP16`, or `FP32`), trading index size against scoring exactness. See <> for the values and their trade-offs. [[hybrid-search]] ==== Hybrid Search with `vector.fuse` diff --git a/src/main/asciidoc/reference/sql/sql-indexes.adoc b/src/main/asciidoc/reference/sql/sql-indexes.adoc index 97d33dbc..4afff53b 100644 --- a/src/main/asciidoc/reference/sql/sql-indexes.adoc +++ b/src/main/asciidoc/reference/sql/sql-indexes.adoc @@ -450,11 +450,29 @@ CREATE PROPERTY Doc.tokens ARRAY_OF_INTEGERS CREATE PROPERTY Doc.weights ARRAY_OF_FLOATS CREATE INDEX ON Doc (tokens, weights) LSM_SPARSE_VECTOR - METADATA { dimensions: 105000, modifier: 'IDF' } + METADATA { dimensions: 105000, modifier: 'IDF', weightQuantization: 'FP32' } ---- The two parallel array properties carry the non-zero dimension ids and matching weights for each record. `dimensions` declares the vocabulary cap and rejects out-of-range ids at write time (set to `0` to disable the cap). `modifier: 'IDF'` enables Robertson-Sparck-Jones IDF weighting at query time; omit it (or set to `'NONE'`) for plain dot-product scoring. +`weightQuantization` (since v26.7.2) controls how posting weights are stored on disk, bringing the sparse index to parity with the dense `LSM_VECTOR` index's quantization knob: + +[%header,cols="1,3"] +|=== +| Value | Storage / trade-off + +| `INT8` +| 1 byte per weight (the default, and the only behavior before v26.7.2). Smallest index; scores are approximate. + +| `FP16` +| 2 bytes per weight. Half the size of `FP32` with near-exact scoring. + +| `FP32` +| 4 bytes per weight. Exact scoring; largest index. Choose this when approximate weights measurably hurt ranking quality. +|=== + +Omit `weightQuantization` to keep the `INT8` default. The setting is fixed at index-creation time; to change it, drop and recreate (or `REBUILD INDEX`) the index. + Query with `vector.sparseNeighbors`: [source,sql] From d43e533a1fb751bede5560b3837ac2a56de364a8 Mon Sep 17 00:00:00 2001 From: robfrank Date: Fri, 10 Jul 2026 12:31:47 +0200 Subject: [PATCH 4/5] docs(#5015): document Cypher write-counter (stats) surfacing over HTTP and gRPC The QueryStatistics write counters (nodes/relationships/properties/labels/ indexes/constraints created/removed, containsUpdates), previously only on Bolt, are now surfaced over HTTP and gRPC, with counts aggregated across UNION branches and CALL subqueries. - http.adoc: add a "Write statistics (stats)" section to the command endpoint documenting the camelCase stats object, its keys, and an example response. Emitted only for write queries. - grpc-api.adoc: document ExecuteCommandResponse and the QueryUpdateStats message (snake_case fields) with a cross-link to the HTTP shape. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/main/asciidoc/reference/grpc-api.adoc | 32 +++++++++++ .../asciidoc/reference/http-api/http.adoc | 56 +++++++++++++++++++ 2 files changed, 88 insertions(+) diff --git a/src/main/asciidoc/reference/grpc-api.adoc b/src/main/asciidoc/reference/grpc-api.adoc index a2fcc31a..6d21090d 100644 --- a/src/main/asciidoc/reference/grpc-api.adoc +++ b/src/main/asciidoc/reference/grpc-api.adoc @@ -127,6 +127,38 @@ The `GrpcValue` message uses a `oneof` to represent any ArcadeDB value: | `InsertOptions` | Transaction mode, conflict handling, and batch size configuration |=== +===== Command Results & Write Statistics + +`ExecuteCommand` returns an `ExecuteCommandResponse`: + +[cols="2,3"] +|=== +| Field | Meaning + +| `success` / `message` | Whether the command succeeded, and an optional message +| `affected_records` | Server-computed count of affected elements and numeric scalars +| `execution_time_ms` | Server-side execution time +| `records` | Optional row payload, returned when `return_rows` is set on the request +| `stats` | A `QueryUpdateStats` message, *present only for write commands that mutated data* (since v26.7.2) +|=== + +`QueryUpdateStats` carries the same write counters the HTTP `stats` object and the Bolt result summary expose — including counts aggregated across OpenCypher `UNION` branches and `CALL { ... }` subqueries: + +[cols="2,3"] +|=== +| Field | Meaning + +| `nodes_created` / `nodes_deleted` | Vertices created / deleted +| `relationships_created` / `relationships_deleted` | Edges created / deleted +| `properties_set` | Properties assigned a value +| `labels_added` / `labels_removed` | Type/label memberships added / removed +| `indexes_added` / `indexes_removed` | Indexes created / dropped +| `constraints_added` / `constraints_removed` | Constraints added / removed +| `contains_updates` | `true` when any counter above is non-zero +|=== + +The equivalent HTTP response object is documented at <>. + ===== Connecting from Python [source,python] diff --git a/src/main/asciidoc/reference/http-api/http.adoc b/src/main/asciidoc/reference/http-api/http.adoc index 1de93c60..f696f28f 100644 --- a/src/main/asciidoc/reference/http-api/http.adoc +++ b/src/main/asciidoc/reference/http-api/http.adoc @@ -966,6 +966,62 @@ $ curl -X POST http://localhost:2480/api/v1/command/mydb \ --user root:arcadedb-password ---- +[[http-write-statistics]] +===== Write statistics (`stats`) + +NOTE: Available since ArcadeDB v26.7.2. + +When a command *modifies* data, the `POST /api/v1/command/{database}` response includes a `stats` object alongside `result`, reporting how many entities the command created, deleted or changed. The object is *omitted for read-only queries* (a command that changes nothing has no `stats` field). It is populated for OpenCypher writes — including counts aggregated across `UNION` branches and `CALL { ... }` subqueries — and mirrors the counters the Bolt driver exposes through its result summary. + +The shape is an ArcadeDB-native camelCase object with a stable set of keys (each counter is always present, `0` when unused): + +[cols="1,3"] +|=== +| Key | Meaning + +| `nodesCreated` / `nodesDeleted` | Vertices created / deleted +| `relationshipsCreated` / `relationshipsDeleted` | Edges created / deleted +| `propertiesSet` | Properties assigned a value +| `labelsAdded` / `labelsRemoved` | Type/label memberships added / removed +| `indexesAdded` / `indexesRemoved` | Indexes created / dropped +| `constraintsAdded` / `constraintsRemoved` | Constraints (`UNIQUE`, `NOT_NULL`, `KEY`, `TYPED`) added / removed +| `containsUpdates` | `true` when any counter above is non-zero +|=== + +Example — a Cypher write returns the counters it produced: + +[source,shell] +---- +$ curl -X POST http://localhost:2480/api/v1/command/school \ + -d '{"language": "cypher", "command": "CREATE (a:Person {name:$n})-[:KNOWS]->(b:Person)", "params": {"n": "Ada"}}' \ + -H "Content-Type: application/json" \ + --user root:arcadedb-password +---- + +[source,json] +---- +{ + "user": "root", + "result": [], + "stats": { + "nodesCreated": 2, + "nodesDeleted": 0, + "relationshipsCreated": 1, + "relationshipsDeleted": 0, + "propertiesSet": 1, + "labelsAdded": 2, + "labelsRemoved": 0, + "indexesAdded": 0, + "indexesRemoved": 0, + "constraintsAdded": 0, + "constraintsRemoved": 0, + "containsUpdates": true + } +} +---- + +The gRPC service carries the same counters in the write-result summary of `ExecuteCommand`; see <>. + [[vector-http-typed-markers]] ===== Typed parameter markers (`$bytes`, `$int8`) From 8fec0fe938e0d655551684e978d743721719fe01 Mon Sep 17 00:00:00 2001 From: robfrank Date: Fri, 10 Jul 2026 13:11:54 +0200 Subject: [PATCH 5/5] docs(26.7.2): recommend static raftStorageDirectory for container/K8s Raft mounts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address review: a dynamically-named raft-storage- directory cannot be mounted as a Docker/Kubernetes volume (a raft-storage-* wildcard is created literally, not expanded) — which is exactly why arcadedb.ha.raftStorageDirectory exists. Update the durable-storage migration note (ha.adoc) and the Docker/Kubernetes upgrade bullet (upgrade.adoc) to set raftStorageDirectory to a static path and mount that, while keeping the plain-filesystem "copy the directory" path. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/main/asciidoc/how-to/operations/ha.adoc | 2 +- src/main/asciidoc/how-to/operations/upgrade.adoc | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/main/asciidoc/how-to/operations/ha.adoc b/src/main/asciidoc/how-to/operations/ha.adoc index c273bcd7..efe90cde 100644 --- a/src/main/asciidoc/how-to/operations/ha.adoc +++ b/src/main/asciidoc/how-to/operations/ha.adoc @@ -334,7 +334,7 @@ With ephemeral Raft storage, wiping the log on restart could leave the cluster i Durable storage lets a restarted node rejoin by *replaying its own persisted log* rather than always requiring a full snapshot resync from the leader. -*Migration:* mount `raft-storage-*` (or the directory named by `arcadedb.ha.raftStorageDirectory`) on durable storage before upgrading. On Kubernetes, back it with a `PersistentVolumeClaim` — see <>. A throwaway or test cluster that intentionally wants ephemeral behavior can opt out with `arcadedb.ha.raftPersistStorage=false`. +*Migration:* the per-node directory is named `raft-storage-` dynamically, which container runtimes and Kubernetes cannot mount as a volume (a wildcard like `raft-storage-*` is created as a literal directory, not expanded). Set `arcadedb.ha.raftStorageDirectory` to a *static* path (e.g. `/home/arcadedb/raft-storage`) and mount *that* path on durable storage before upgrading. On Kubernetes, back it with a `PersistentVolumeClaim` — see <>. On a plain filesystem you can instead copy the existing `raft-storage-` directory across the upgrade. A throwaway or test cluster that intentionally wants ephemeral behavior can opt out with `arcadedb.ha.raftPersistStorage=false`. [[ha-remote-commit-contract]] ===== Committed-remotely response contract (from v26.7.2) diff --git a/src/main/asciidoc/how-to/operations/upgrade.adoc b/src/main/asciidoc/how-to/operations/upgrade.adoc index dbd8eb68..6bd97854 100644 --- a/src/main/asciidoc/how-to/operations/upgrade.adoc +++ b/src/main/asciidoc/how-to/operations/upgrade.adoc @@ -153,7 +153,7 @@ The only data that survives across container restarts is what lives on a persist * Mount `/home/arcadedb/databases` on a persistent volume — this is the equivalent of "copy the `databases/` folder". * Mount `/home/arcadedb/config` only if you have edited the files listed in <>; otherwise let the image ship its defaults. -* Mount `/home/arcadedb/raft-storage-*` (or the path set by `arcadedb.ha.raftStorageDirectory`) on a persistent volume. Since v26.7.2 the Raft log is durable by default (`arcadedb.ha.raftPersistStorage=true`) so a restarted pod can replay its own log instead of resyncing from peers. On releases before v26.7.2, or with `raftPersistStorage=false`, the log is ephemeral and this mount is unnecessary. +* Set `arcadedb.ha.raftStorageDirectory` to a static path (e.g. `/home/arcadedb/raft-storage`) and mount *that* path on a persistent volume. The default per-node directory is named `raft-storage-` dynamically, which a container runtime or Kubernetes cannot mount as a volume (a wildcard like `/home/arcadedb/raft-storage-*` is created literally, not expanded) — this is exactly what `raftStorageDirectory` is for. Since v26.7.2 the Raft log is durable by default (`arcadedb.ha.raftPersistStorage=true`) so a restarted pod can replay its own log instead of resyncing from peers. On releases before v26.7.2, or with `raftPersistStorage=false`, the log is ephemeral and this mount is unnecessary. * Mount `/home/arcadedb/backups` if you want to retain backup archives across pod restarts. For Kubernetes specifics (StatefulSet, headless service, init container for pre-staging a database), see <>.