Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions src/main/asciidoc/concepts/vector-search.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
};
----

The factory rejects two misconfigurations at index-creation time so the failure surfaces immediately rather than as silent mis-scaling later:
The factory rejects two misconfigurations at index-creation time so the failure surfaces immediately rather than as silent incorrect scaling later:

* `encoding=INT8` with a non-`BINARY` property (or `encoding=FLOAT32` with a `BINARY` property) -- the property type and encoding must agree.
* `encoding=INT8` combined with `quantization=INT8` -- redundant: the property is already byte-quantized at the wire level, so the index-internal scalar quantizer would re-quantize the dequantized floats. Pick one (`encoding=INT8` for payload/storage savings, `quantization=INT8` for index-internal compression), not both.
Expand Down Expand Up @@ -505,7 +505,9 @@ Options map (5th argument) supports `filter` (allowed-RIDs whitelist), `groupBy`

Retrieval uses document-at-a-time WAND with per-dim `max_weight` upper bounds: per-dim cursors traverse postings in RID order, the algorithm pivots to the smallest RID whose cumulative upper bound can still beat the current K-th best score, and skips cursors below the pivot via direct seeks. Only dot-product similarity is supported (cosine on sparse vectors is conventionally handled by L2-normalizing both the query and stored vectors at insert time).

The MVP works well up to roughly 10M sparse vectors. The scaling story past that point (BlockMax-WAND with per-page bounds, weight quantization, parallel per-segment scoring) is tracked separately.
The MVP works well up to roughly 10M sparse vectors. The scaling story past that point (BlockMax-WAND with per-page bounds, parallel per-segment scoring) is tracked separately.

Posting-weight quantization is configurable since v26.7.2 via the `weightQuantization` index `METADATA` key (`INT8` — the default — `FP16`, or `FP32`), trading index size against scoring exactness. See <<sql-create-index,`CREATE INDEX`>> for the values and their trade-offs.

[[hybrid-search]]
==== Hybrid Search with `vector.fuse`
Expand Down
6 changes: 6 additions & 0 deletions src/main/asciidoc/how-to/connectivity/bolt.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -420,6 +420,7 @@ try (Session read = driver.session(SessionConfig.builder()
}
----

[[bolt-datatypes]]
===== Data Type Mapping

ArcadeDB emits native BOLT/PackStream structures for the following, so drivers receive typed objects (not strings):
Expand All @@ -434,6 +435,11 @@ ArcadeDB emits native BOLT/PackStream structures for the following, so drivers r

Values without a native BOLT type — `RID` and `UUID` — are serialized as strings. `BigDecimal` and out-of-range `BigInteger` values are converted to `double` and may lose precision.

[NOTE]
====
*Changed in v26.7.2 (breaking for Bolt clients).* Temporal values (`Date`, `Time`, `LocalTime`, `DateTime`, `LocalDateTime`) are now carried as native PackStream temporal structures in *both* directions. Before v26.7.2 they were sent as ISO-8601 strings, and inbound datetime query parameters were silently dropped. A client that previously read a temporal property as a `String` must now read the native temporal type — e.g. `Value.asZonedDateTime()` or `Value.asLocalDate()` — exactly as it would against Neo4j. See <<upgrade-breaking-changes,Upgrade — Breaking changes>>.
====

===== Error Codes

Server errors are mapped to Neo4j-style structured status codes so driver-side error handling and retry logic behave as expected:
Expand Down
10 changes: 7 additions & 3 deletions src/main/asciidoc/how-to/operations/groups.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@ Where:
The wildcard "`*`" represents all the types that are not defined in this configuration.
* `access` is the array containing the allowed permissions for the group.
The supported permission at group level are:
** `updateSecurity`: to update the security settings (create, modify and delete users, groups, etc.)
** `updateSecurity`: to update the security settings (create, modify and delete users, groups, etc.). Since v26.7.2 this also gates defining a JavaScript function via `DEFINE FUNCTION ... LANGUAGE js` — see <<user-functions,User Functions>>.
** `updateSchema`: to update the database schema (create, modify and drop buckets, types and indexes)
** `updateDatabaseSettings` (since v26.7.2): to change per-database settings — e.g. `ALTER DATABASE`, `ALTER TYPE ... BUCKETSELECTIONSTRATEGY`, materialized-view / continuous-aggregate refresh and downsampling policies. Previously several of these mutators were reachable without a dedicated privilege.
* `readTimeout` if present, specify the maximum timeout for read operations. -1 means no limits.
If set, all the read operations (lookups and queries) will be limited to maximum `<readTimeout>` milliseconds.
This is useful to limit users to execute expensive commands and queries impacting the performance of the server and therefore other connected users.
Expand Down Expand Up @@ -71,7 +72,8 @@ The default settings for the `admin` group are:
{
"access": [
"updateSecurity",
"updateSchema"
"updateSchema",
"updateDatabaseSettings"
],
"resultSetLimit": -1,
"readTimeout": -1,
Expand All @@ -88,7 +90,9 @@ The default settings for the `admin` group are:
}
```

Which allows to execute any operation against the security, the schema and records.
Which allows to execute any operation against the security, the schema, the database settings and records.

NOTE: The `updateDatabaseSettings` permission was added in v26.7.2. When ArcadeDB opens an older `server-groups.json`, it automatically migrates existing `admin` groups to include it, so upgraded clusters keep full administrative access without a manual edit.

Here is an example for an append-only group:

Expand Down
34 changes: 32 additions & 2 deletions src/main/asciidoc/how-to/operations/ha.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,32 @@ Common causes of leader unavailability include:

When a replica rejoins after being offline, it catches up via the Raft log automatically; if it has fallen behind past the log purge boundary, the leader streams a full database snapshot over HTTP and the replica installs it atomically.

Since v26.7.2, the leader also *auto-recovers a follower's replication channel* that is wedged on a stale peer address — for example after a Kubernetes pod is rescheduled onto a new IP. Previously such a follower stayed silently disconnected and the cluster ran at bare quorum until an operator forced a leadership transfer; the leader now re-resolves the peer and re-establishes the channel on its own.

[[ha-durable-raft-storage]]
===== Durable Raft Storage (from v26.7.2)

[IMPORTANT]
====
Since v26.7.2, `arcadedb.ha.raftPersistStorage` defaults to `true` (it was `false` before). The per-node Raft log under `raft-storage-<nodeName>` is now *persisted across restarts*, and `arcadedb.ha.raftStorageDirectory` must live on *durable* storage (a persistent volume, not an ephemeral or `tmpfs` mount).
====

With ephemeral Raft storage, wiping the log on restart could leave the cluster in a broken state:

* a lagging follower that came back after a *full-cluster cold restart* could permanently diverge and fail to rejoin, surfacing as a `WALVersionGapException`; or
* the cluster could silently re-form as a fresh single-node cluster, dropping committed history.

Durable storage lets a restarted node rejoin by *replaying its own persisted log* rather than always requiring a full snapshot resync from the leader.

*Migration:* the per-node directory is named `raft-storage-<nodeName>` dynamically, which container runtimes and Kubernetes cannot mount as a volume (a wildcard like `raft-storage-*` is created as a literal directory, not expanded). Set `arcadedb.ha.raftStorageDirectory` to a *static* path (e.g. `/home/arcadedb/raft-storage`) and mount *that* path on durable storage before upgrading. On Kubernetes, back it with a `PersistentVolumeClaim` — see <<kubernetes,Kubernetes>>. On a plain filesystem you can instead copy the existing `raft-storage-<nodeName>` directory across the upgrade. A throwaway or test cluster that intentionally wants ephemeral behavior can opt out with `arcadedb.ha.raftPersistStorage=false`.

[[ha-remote-commit-contract]]
===== Committed-remotely response contract (from v26.7.2)

When a write is forwarded from a replica to the leader (or issued directly to the leader), it is possible for the transaction to be *committed cluster-wide by quorum* and then fail to apply on the node that is answering the client. This is a distinct outcome from an ordinary failure: the data is durably committed, so retrying would create a duplicate.

Since v26.7.2 this case has its own contract. The server raises a `TransactionCommittedRemotelyException`, mapped over HTTP to *status `409 Conflict`*. A client that receives it must *not* retry the transaction — the write already succeeded cluster-wide; it should instead re-read to observe the committed state. This is different from a transient conflict (a genuine write-write conflict or `DeadlockException`), which remains safe to retry.

[[ha-offline-bootstrap]]
===== Offline Cluster Bootstrap (from v26.5.1)

Expand Down Expand Up @@ -546,8 +572,12 @@ A complete list of all HA parameters is available in <<arcadedb-settings,Server
|10000 / 100

|`+arcadedb.ha.raftPersistStorage+`
|If true, the Raft storage directory is preserved across restarts, enabling rejoin without a full snapshot resync
|false
|If true, the Raft storage directory is preserved across restarts, enabling rejoin by replaying the persisted log instead of a full snapshot resync. *Defaults to `true` since v26.7.2* (previously `false`); ensure `raftStorageDirectory` lives on durable storage. See <<ha-durable-raft-storage,Durable Raft Storage>>
|true

|`+arcadedb.ha.raftStorageDirectory+`
|Parent directory for the per-node `raft-storage-<nodeName>` sub-folder. Empty (default) uses the server root path. With durable storage enabled by default, point this at a persistent volume. See <<ha-durable-raft-storage,Durable Raft Storage>>
|(empty)

|`+arcadedb.ha.stopServerOnReplicationFailure+`
|If true, the JVM exits after exhausting step-down retries on a phase-2 replication failure. Default false keeps the server up and logs CRITICAL
Expand Down
31 changes: 25 additions & 6 deletions src/main/asciidoc/how-to/operations/upgrade.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,24 @@ The migration is transparent and happens lazily on first open, so the only opera

This section lists behavior changes that may require action when upgrading.

[discrete]
====== 26.7.2 — Durable Raft storage is now the default

`arcadedb.ha.raftPersistStorage` now defaults to `true` (previously `false`). The per-node Raft log under `raft-storage-<nodeName>` is persisted across restarts so a node can rejoin by replaying its own log instead of always requiring a full snapshot resync. Wiping the log on restart could previously diverge a lagging follower after a full-cluster cold restart (`WALVersionGapException`) or silently re-form a fresh single-node cluster.

*Impact:* HA clusters only.

*Action:* ensure `arcadedb.ha.raftStorageDirectory` (or the default `raft-storage-*` location under the server root) lives on *durable* storage — a persistent volume, not an ephemeral or `tmpfs` mount. On Kubernetes, back it with a `PersistentVolumeClaim`. A throwaway/test cluster can opt out with `arcadedb.ha.raftPersistStorage=false`. See <<ha-durable-raft-storage,Durable Raft Storage>>.

[discrete]
====== 26.7.2 — Bolt temporal values are now native PackStream types

Over the Neo4j Bolt protocol, `date`, `time`, `localtime`, `datetime` and `localdatetime` are now carried as *native PackStream temporal structures* in both directions, matching Neo4j. Previously they were serialized as ISO-8601 strings (and inbound datetime query parameters were silently dropped).

*Impact:* Bolt/Neo4j-driver clients that read a temporal property as a `String`.

*Action:* read temporal properties with the driver's native temporal accessors (e.g. `Value.asZonedDateTime()`, `asLocalDate()`), exactly as you would against Neo4j. See <<bolt-datatypes,Bolt data type mapping>>.

[discrete]
====== 26.7.1 — Cypher `IS TYPED INT` is now 32-bit (`INTEGER`)

Expand Down Expand Up @@ -55,8 +73,8 @@ Only a few of them hold user-owned state; the rest are recreated from the releas
|Server log files. Safe to start with an empty `log/` directory; the release ships an empty one. Copy only if you want to keep historical logs alongside the new install.

|`raft-storage-<peerId>/` (HA only)
|Skip
|Per-node Raft log segments, created at runtime when HA is enabled. By default this directory is recreated on every restart, so there is nothing to migrate. See <<upgrade-ha-cluster,HA cluster upgrade>> for details.
|*Copy* (since v26.7.2)
|Per-node Raft log segments, created at runtime when HA is enabled. Since v26.7.2 this storage is *durable by default* (`arcadedb.ha.raftPersistStorage=true`) and must be preserved so a node can rejoin by replaying its log instead of a full snapshot resync. On releases before v26.7.2, or when `raftPersistStorage` is explicitly `false`, it is ephemeral and can be skipped. See <<upgrade-ha-cluster,HA cluster upgrade>> for details.

|`replication/`
|Skip
Expand Down Expand Up @@ -110,9 +128,10 @@ What to migrate on each node:
* *`databases/`* — copy, exactly as in the single-server case.
* *`config/`* — copy the same files as in the single-server case.
The cluster's identity is derived from `arcadedb.ha.clusterName` and the `root` password (the inter-node token is computed from them at startup), so as long as every node keeps the same cluster name and the same root password, the upgraded cluster reattaches to itself without any extra step.
* *`raft-storage-<peerId>/`* — *do not copy*.
This directory holds the local Raft log and is wiped and re-created on every restart.
After the upgraded node rejoins the cluster, the leader replays missing entries via Raft or, if the node has fallen behind the purge boundary, streams a full snapshot over HTTP automatically.
* *`raft-storage-<peerId>/`* — *copy* (since v26.7.2).
Since v26.7.2 the local Raft log is durable by default (`arcadedb.ha.raftPersistStorage=true`), so preserve this directory across the upgrade to let the node rejoin by replaying its own log.
Whether or not it is preserved, after the upgraded node rejoins the cluster the leader replays any missing entries via Raft or, if the node has fallen behind the purge boundary, streams a full snapshot over HTTP automatically.
On releases before v26.7.2 (or with `raftPersistStorage=false`) this directory is ephemeral and can be skipped.
* *`backups/` and `log/`* — same as single-server.

Recommended rolling upgrade flow (zero downtime, requires quorum at all times):
Expand All @@ -134,7 +153,7 @@ The only data that survives across container restarts is what lives on a persist

* Mount `/home/arcadedb/databases` on a persistent volume — this is the equivalent of "copy the `databases/` folder".
* Mount `/home/arcadedb/config` only if you have edited the files listed in <<upgrade-config-files>>; otherwise let the image ship its defaults.
* Do *not* mount `/home/arcadedb/raft-storage-*` on a persistent volume; the Raft log is ephemeral and is rebuilt from peers on every pod restart.
* Set `arcadedb.ha.raftStorageDirectory` to a static path (e.g. `/home/arcadedb/raft-storage`) and mount *that* path on a persistent volume. The default per-node directory is named `raft-storage-<nodeName>` dynamically, which a container runtime or Kubernetes cannot mount as a volume (a wildcard like `/home/arcadedb/raft-storage-*` is created literally, not expanded) — this is exactly what `raftStorageDirectory` is for. Since v26.7.2 the Raft log is durable by default (`arcadedb.ha.raftPersistStorage=true`) so a restarted pod can replay its own log instead of resyncing from peers. On releases before v26.7.2, or with `raftPersistStorage=false`, the log is ephemeral and this mount is unnecessary.
* Mount `/home/arcadedb/backups` if you want to retain backup archives across pod restarts.

For Kubernetes specifics (StatefulSet, headless service, init container for pre-staging a database), see <<kubernetes,Kubernetes>>.
Expand Down
32 changes: 32 additions & 0 deletions src/main/asciidoc/reference/grpc-api.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,38 @@ The `GrpcValue` message uses a `oneof` to represent any ArcadeDB value:
| `InsertOptions` | Transaction mode, conflict handling, and batch size configuration
|===

===== Command Results & Write Statistics

`ExecuteCommand` returns an `ExecuteCommandResponse`:

[cols="2,3"]
|===
| Field | Meaning

| `success` / `message` | Whether the command succeeded, and an optional message
| `affected_records` | Server-computed count of affected elements and numeric scalars
| `execution_time_ms` | Server-side execution time
| `records` | Optional row payload, returned when `return_rows` is set on the request
| `stats` | A `QueryUpdateStats` message, *present only for write commands that mutated data* (since v26.7.2)
|===

`QueryUpdateStats` carries the same write counters the HTTP `stats` object and the Bolt result summary expose — including counts aggregated across OpenCypher `UNION` branches and `CALL { ... }` subqueries:

[cols="2,3"]
|===
| Field | Meaning

| `nodes_created` / `nodes_deleted` | Vertices created / deleted
| `relationships_created` / `relationships_deleted` | Edges created / deleted
| `properties_set` | Properties assigned a value
| `labels_added` / `labels_removed` | Type/label memberships added / removed
| `indexes_added` / `indexes_removed` | Indexes created / dropped
| `constraints_added` / `constraints_removed` | Constraints added / removed
| `contains_updates` | `true` when any counter above is non-zero
|===

The equivalent HTTP response object is documented at <<http-write-statistics,Write statistics>>.

===== Connecting from Python

[source,python]
Expand Down
56 changes: 56 additions & 0 deletions src/main/asciidoc/reference/http-api/http.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -966,6 +966,62 @@ $ curl -X POST http://localhost:2480/api/v1/command/mydb \
--user root:arcadedb-password
----

[[http-write-statistics]]
===== Write statistics (`stats`)

NOTE: Available since ArcadeDB v26.7.2.

When a command *modifies* data, the `POST /api/v1/command/{database}` response includes a `stats` object alongside `result`, reporting how many entities the command created, deleted or changed. The object is *omitted for read-only queries* (a command that changes nothing has no `stats` field). It is populated for OpenCypher writes — including counts aggregated across `UNION` branches and `CALL { ... }` subqueries — and mirrors the counters the Bolt driver exposes through its result summary.

The shape is an ArcadeDB-native camelCase object with a stable set of keys (each counter is always present, `0` when unused):

[cols="1,3"]
|===
| Key | Meaning

| `nodesCreated` / `nodesDeleted` | Vertices created / deleted
| `relationshipsCreated` / `relationshipsDeleted` | Edges created / deleted
| `propertiesSet` | Properties assigned a value
| `labelsAdded` / `labelsRemoved` | Type/label memberships added / removed
| `indexesAdded` / `indexesRemoved` | Indexes created / dropped
| `constraintsAdded` / `constraintsRemoved` | Constraints (`UNIQUE`, `NOT_NULL`, `KEY`, `TYPED`) added / removed
| `containsUpdates` | `true` when any counter above is non-zero
|===

Example — a Cypher write returns the counters it produced:

[source,shell]
----
$ curl -X POST http://localhost:2480/api/v1/command/school \
-d '{"language": "cypher", "command": "CREATE (a:Person {name:$n})-[:KNOWS]->(b:Person)", "params": {"n": "Ada"}}' \
-H "Content-Type: application/json" \
--user root:arcadedb-password
----

[source,json]
----
{
"user": "root",
"result": [],
"stats": {
"nodesCreated": 2,
"nodesDeleted": 0,
"relationshipsCreated": 1,
"relationshipsDeleted": 0,
"propertiesSet": 1,
"labelsAdded": 2,
"labelsRemoved": 0,
"indexesAdded": 0,
"indexesRemoved": 0,
"constraintsAdded": 0,
"constraintsRemoved": 0,
"containsUpdates": true
}
}
----

The gRPC service carries the same counters in the write-result summary of `ExecuteCommand`; see <<grpc-api,gRPC API>>.

[[vector-http-typed-markers]]
===== Typed parameter markers (`$bytes`, `$int8`)

Expand Down
Loading
Loading