Skip to content
Merged
105 changes: 105 additions & 0 deletions .fluree-memory/repo.ttl
Original file line number Diff line number Diff line change
Expand Up @@ -1461,6 +1461,22 @@ mem:constraint-01kwjtjew74a727brs0q3jaf7e a mem:Constraint ;
mem:branch "feature/rdfs-enforcement-entailment" ;
mem:createdAt "2026-07-03T01:47:55.655870+00:00"^^xsd:dateTime .

mem:decision-01kwm1agwk619zyf6rsxrjj7ap a mem:Decision ;
mem:content "Remote-mount / query-peer feature v1 scope: ledger access over the block-serving (peer) path is all-or-nothing per (token, ledger) — full-access raw CAS bytes (CID-verifiable, globally cacheable) or nothing; plus a public/anonymous tier. No policy-filtered (FLKB) leaf serving to peers in v1 — fine-grained permissions require query-shipping (server executes with row-level policy). Requires adding a full-access raw mode to the storage proxy (EnforcementMode::TrustedInternal gated on token scope; today PolicyEnforced/FLKB is the only proxy mode). v2 hooks: FLKB negotiation stays in ProxyStorage, serving advertisement uses distinct \"blocks\" vs future \"blocks:filtered\", cache writes carry a CID-verified flag." ;
mem:tag "caching" ;
mem:tag "feature-design" ;
mem:tag "peer-mode" ;
mem:tag "policy" ;
mem:tag "remote-mounts" ;
mem:tag "storage-proxy" ;
mem:scope mem:repo ;
mem:artifactRef "fluree-db-api/src/block_fetch.rs" ;
mem:artifactRef "fluree-db-server/src/peer/proxy_storage.rs" ;
mem:artifactRef "fluree-db-server/src/routes/storage_proxy.rs" ;
mem:branch "feature/rdfs-enforcement-entailment" ;
mem:createdAt "2026-07-03T13:05:10.035196+00:00"^^xsd:dateTime ;
mem:rationale "Avoids the two hardest v1 problems: filtered leaves are not immutable (policy changes make cached filtered views a revocation leak) and non-byte-identical payloads break CAS integrity checks. All-or-nothing keeps every peer-fetched block canonical and cache-forever." .

mem:fact-01kwjtjbnvb36dtaf2zrq806cj a mem:Fact ;
mem:content "RDFS enforcement entailment (feature/rdfs-enforcement-entailment): always-on subclass+subproperty inference for SHACL and Policy. Epoch counters on Novelty bumped in apply_commit's routing loop (schema_epoch: subClassOf/subPropertyOf; shacl_epoch: sh:* predicates or rdf:type→SHACL-type/rdfs:Class/owl:Class) + two Arc caches on LedgerState carried across commits: SchemaHierarchyCache (core; keyed indexed-schema-t + schema_epoch) and a type-erased compiled-SHACL slot (+shacl_epoch +shapes_g_ids; data-only txns skip ShapeCompiler via ShaclEngine::from_shared_cache). RULE: enforcement uses COMMITTED hierarchy — same-txn schema does not entail (two-txn workflow, pinned by test)." ;
mem:tag "enforcement" ;
Expand Down Expand Up @@ -1490,6 +1506,95 @@ mem:fact-01kws806qkz7h5w609y1b0r25n a mem:Fact ;
mem:branch "feature/shacl-property-paths" ;
mem:createdAt "2026-07-05T13:38:04.147463+00:00"^^xsd:dateTime .

mem:fact-01kwktkb2r0ktxt5xaxvbwev3e a mem:Fact ;
mem:content "The \"query peer over HTTP\" model exists as server peer proxy mode: ProxyStorage + ProxyNameService wired via FlureeBuilder::memory().build_with(storage, NameServiceMode::ReadOnly); serving side is the StorageProxyBearer route group; GET /nameservice/snapshot filters to the token's fluree.storage.ledgers scope. ProxyStorage is mode-aware (feature/remote-mounts): ProxyReadMode::Raw fetches canonical CID-verified bytes via GET /storage/objects/{cid} (peer default; what makes indexed ledgers readable), ProxyReadMode::Filtered uses POST /storage/block FLKB negotiation. CRITICAL: no production code decodes FLKB — the filtered tier is transport-only until a leaf decoder exists on the read path. Remaining gaps: one upstream per server (no composite nameservice), no HTTP Range on ProxyStorage, no vended-credential/origin handoff." ;
mem:tag "auth" ;
mem:tag "federation" ;
mem:tag "nameservice" ;
mem:tag "peer-mode" ;
mem:tag "remote" ;
mem:tag "storage-proxy" ;
mem:scope mem:repo ;
mem:artifactRef "fluree-db-server/src/peer/proxy_nameservice.rs" ;
mem:artifactRef "fluree-db-server/src/peer/proxy_storage.rs" ;
mem:artifactRef "fluree-db-server/src/routes/nameservice_refs.rs" ;
mem:artifactRef "fluree-db-server/src/routes/storage_proxy.rs" ;
mem:artifactRef "fluree-db-server/src/state.rs" ;
mem:branch "feature/rdfs-enforcement-entailment" ;
mem:createdAt "2026-07-03T11:07:38.968674+00:00"^^xsd:dateTime ;
mem:rationale "Any future \"connect to external Fluree as a data source\" feature should extend peer proxy mode rather than invent a new graph-source kind; re-discovering this subsystem map cost four parallel explorations." .

mem:constraint-01kwm1fjv37hjcfsga916mc5wv a mem:Constraint ;
mem:content "docs/design/ files must document permanent design — what exists and how it works — never phased plans, roadmaps, or implementation sequencing. Planning content goes in a temporary file elsewhere (scratchpad or uncommitted), not in docs/design/." ;
mem:tag "design-docs" ;
mem:tag "docs" ;
mem:tag "user-feedback" ;
mem:tag "workflow" ;
mem:scope mem:repo ;
mem:severity "must" ;
mem:branch "feature/remote-mounts" ;
mem:createdAt "2026-07-03T13:07:55.875908+00:00"^^xsd:dateTime ;
mem:rationale "User feedback when starting the remote-mounts feature: a proposed design doc had drifted toward a work plan (implementation slices/phases). Same spirit as the existing rule against plan labels in code comments and commit bodies." .

mem:fact-01kwm585khra36js0r49n36k06 a mem:Fact ;
mem:content "Tests that run a peer-mode Fluree query against an in-process axum server must use #[tokio::test(flavor = \"multi_thread\")]: lazy dict-blob materialization during query execution bridges sync→async (block_on), which starves a current-thread runtime so the in-process server can never answer the HTTP fetch — the ProxyStorage request times out after its full 60s. See test_peer_proxy_fluree_queries_indexed_ledger." ;
mem:tag "gotcha" ;
mem:tag "peer-mode" ;
mem:tag "sync-bridge" ;
mem:tag "testing" ;
mem:tag "tokio" ;
mem:scope mem:repo ;
mem:artifactRef "fluree-db-server/tests/proxy_integration.rs" ;
mem:branch "feature/remote-mounts" ;
mem:createdAt "2026-07-03T14:13:47.249260+00:00"^^xsd:dateTime .

mem:fact-01kwm9svy064yer4ek7rn251hh a mem:Fact ;
mem:content "Per-ledger serving posture = f:servingDefaults setting group (f:serveQuery/f:serveBlocks/f:publicVisibility, ledger-scoped, LedgerConfig.serving) enforced only on transaction-role servers: query gate in load_ledger_for_query (403 stable msg), blocks gate in storage_proxy/pack/commits (404). GOTCHA: @shared dict-blob addresses carry only the ledger NAME, so ProxyStorage derives a default-branch alias — any server-side per-ledger guard on block/object endpoints must branch-resolve DictBlob requests via list_branches (resolve_block_ledger in storage_proxy.rs) or non-main-branch peers break on dict fetches. Legacy per-branch dict layout ({name}/{branch}/index/objects/dicts) is also parsed by cid_and_ledger_from_address." ;
mem:tag "config-graph" ;
mem:tag "dict-blob" ;
mem:tag "gotcha" ;
mem:tag "peer-mode" ;
mem:tag "serving-defaults" ;
mem:tag "storage-proxy" ;
mem:scope mem:repo ;
mem:artifactRef "fluree-db-api/src/config_resolver.rs" ;
mem:artifactRef "fluree-db-core/src/ledger_config.rs" ;
mem:artifactRef "fluree-db-server/src/routes/serving.rs" ;
mem:artifactRef "fluree-db-server/src/routes/storage_proxy.rs" ;
mem:branch "feature/remote-mounts" ;
mem:createdAt "2026-07-03T15:33:21.472958+00:00"^^xsd:dateTime .

mem:fact-01kwmb5yq11y50j3kbd7c5tne4 a mem:Fact ;
mem:content "Remote mounts: FlureeBuilder::with_remote_mount(RemoteMountSpec) applies in finalize_with_backend → CompositeNameService (prefix-routed reads, records localized to \"acme/inventory:main\", writes to mounts rejected, full publisher surface) + StorageBackend::Routed (namespace-prefix → mount backend at the content_store seam; admin ops = default only). ProxyStorage/ProxyNameService live in fluree-db-nameservice-sync (server peer/* re-exports); ProxyStorage::with_local_prefix strips the mount prefix from derived aliases. Transact writes to mounts fail at routed ProxyStorage (read-only) before the NS guard — the guard backstops create/branch/publish." ;
mem:tag "builder" ;
mem:tag "composite-nameservice" ;
mem:tag "peer-mode" ;
mem:tag "remote-mounts" ;
mem:tag "storage-backend" ;
mem:scope mem:repo ;
mem:artifactRef "fluree-db-api/src/lib.rs" ;
mem:artifactRef "fluree-db-core/src/storage.rs" ;
mem:artifactRef "fluree-db-nameservice-sync/src/proxy_storage.rs" ;
mem:artifactRef "fluree-db-nameservice/src/mount.rs" ;
mem:artifactRef "fluree-db-server/tests/proxy_integration.rs" ;
mem:branch "feature/remote-mounts" ;
mem:createdAt "2026-07-03T15:57:26.113125+00:00"^^xsd:dateTime .

mem:fact-01kws95166gqpjjfda0nvmkraq a mem:Fact ;
mem:content "Any per-(ledger,t) cache of config-derived server state MUST key on the novelty-inclusive t (LedgerState::t() / state.t()), NOT the indexed base snapshot.snapshot.t. A config-graph change (e.g. f:servingDefaults) commits into novelty and bumps the effective t WITHOUT advancing the indexed base t until a reindex. Keying on the base t serves a stale posture across config changes. This exact bug hit AppState::effective_serving_cached and was caught by test_serving_defaults_gate_and_advertisement (proxy_integration.rs): got 200 expecting 404 because the cached entry didn't invalidate. resolve_ledger_config / effective_serving_from_state resolve at state.t(), so the cache key must match." ;
mem:tag "cache" ;
mem:tag "config-graph" ;
mem:tag "novelty" ;
mem:tag "server" ;
mem:tag "serving" ;
mem:scope mem:repo ;
mem:artifactRef "fluree-db-api/src/config_resolver.rs" ;
mem:artifactRef "fluree-db-server/src/routes/serving.rs" ;
mem:artifactRef "fluree-db-server/src/state.rs" ;
mem:branch "feature/remote-mounts" ;
mem:createdAt "2026-07-05T13:58:10.886197+00:00"^^xsd:dateTime ;
mem:rationale "Non-obvious: indexed snapshot t and effective (novelty-inclusive) t diverge between a commit and the next reindex; config changes live in novelty, so t-keyed caches of config-derived state silently go stale if keyed on the base t." .

mem:fact-01kvzn4mmmcvc7zz6bcswtw47c a mem:Fact ;
mem:content "SPARQL property paths, branch feature/sparql-property-paths: closed ALL reported gaps PLUS inverse-in-composite (wired in BOTH sparql lower/path.rs AND query parse/lower.rs for JSON-LD parity). (1) ZeroOrOne p?: PathModifier::ZeroOrOne; BFS self+1hop. (2) Negated sets !: fresh ?__np{n} pred-var triple + FILTER(?p != IRI(...)); fwd/inv/mixed=UNION. (3) Nested modifiers (p*)*: collapse_modifiers (zero=any */?, unbounded=any +/*). (4) Composite-transitive (a/b)+ incl inverse steps (^a/b)+: PropertyPathPattern has predicates+first_inverse+sequence_steps:Vec<PathStep{predicates,inverse}>; operator's read_step reads SPOT/POST by direction (opposite index when retreating); single_predicate() bails on composite. ONLY UNSUPPORTED (niche): nested transitive step in a composite unit (a+/b)+ — not exercised by any W3C test." ;
mem:tag "negated-set" ;
Expand Down
4 changes: 4 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ tokio = { version = "1", features = ["full", "test-util"] }
aws-config = "1.8.15"
aws-sdk-s3 = "1.124.0"
aws-sdk-dynamodb = "1.109.0"
aws-sdk-sts = "1.101.0"
aws-credential-types = "1.2"
aws-smithy-types = "1.4.7"
# HTTP/1.1-pinned transport for the S3 SDK. GCS's S3-interop endpoint negotiates
Expand Down
3 changes: 3 additions & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
- [publish](cli/publish.md)
- [clone](cli/clone.md)
- [track](cli/track.md)
- [cache](cli/cache.md)
- [server](cli/server.md)
- [memory](cli/memory.md)
- [mcp](cli/mcp.md)
Expand Down Expand Up @@ -73,6 +74,7 @@
- [Time travel patterns](guides/cookbook-time-travel.md)
- [Branching and merging](guides/cookbook-branching.md)
- [Access control policies](guides/cookbook-policies.md)
- [Sharing data with downstream consumers](guides/sharing-data.md)
- [SHACL validation](guides/cookbook-shacl.md)
- [Edge annotations](guides/cookbook-edge-annotations.md)
- [`owl:imports` across named graphs](guides/cookbook-owl-imports.md)
Expand All @@ -82,6 +84,7 @@
- [Auth contract (CLI ↔ Server)](design/auth-contract.md)
- [Nameservice schema v2](design/nameservice-schema-v2.md)
- [Storage-agnostic commits and sync](design/storage-agnostic-commits-and-sync.md)
- [Remote mounts and serving tiers](design/remote-mounts.md)
- [ContentId and ContentStore](design/content-id-and-contentstore.md)
- [Index format](design/index-format.md)
- [Edge annotations (storage internals)](design/edge-annotations.md)
Expand Down
54 changes: 54 additions & 0 deletions docs/api/endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -911,6 +911,60 @@ curl -H "Authorization: Bearer $TOKEN" \
"http://localhost:8090/v1/fluree/storage/objects/bafybeig...leafCid?ledger=mydb:main"
```

Single-range `Range: bytes=start-end` requests are honored with `206 Partial
Content` (`Content-Range` included); the full object is verified against the
CID before slicing. Unsupported Range forms fall back to a full `200`
response; a start at or past the object length returns `416`.

### GET /storage/credentials

Mint STS credentials scoped to a ledger's S3 prefix, so an authorized peer
reads index content **directly from S3** instead of proxying every object
through this server. See [Remote mounts and serving
tiers](../design/remote-mounts.md).

**URL:**

```
GET /storage/credentials?ledger={ledger-id}
```

**Requirements (all must hold, else `404`):**

- Server built with the `aws` feature and **single-bucket S3-backed storage**
(connection config)
- `--storage-vend-enabled` and `--storage-vend-role-arn <arn>` configured
- Bearer token with `fluree.storage.*` scope covering the ledger
- The ledger's `f:serveBlocks` posture allows raw serving

**Response Body** (200 OK):

```json
{
"access_key_id": "ASIA...",
"secret_access_key": "...",
"session_token": "...",
"expires_at_epoch_secs": 1751600000,
"bucket": "my-fluree-bucket",
"region": "us-east-1",
"key_prefix": "ledgers",
"scoped_prefix": "ledgers/mydb"
}
```

The grant is an STS `AssumeRole` session narrowed by a session policy to
`s3:GetObject` under the ledger's **name-level** prefix (covering all
branches and the shared dictionary namespace) plus prefix-conditioned
`s3:ListBucket`. TTL is `--storage-vend-ttl-secs` (default 900, the STS
minimum).

**Revocation caveat:** a minted grant stays valid until it expires — token
revocations and `f:serveBlocks` changes take effect at grant expiry, so keep
TTLs short.

Consumers (CLI peer mode, `fluree-db-nameservice-sync::vended_s3`) probe this
endpoint and fall back to proxied `/storage/objects` reads on `404`.

## Nameservice Sync Endpoints

Used by replication clients and peer instances to push ref updates, initialize
Expand Down
41 changes: 41 additions & 0 deletions docs/cli/cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# fluree cache

Inspect or clear the peer-mode index artifact cache.

When a tracked ledger uses `--mode peer` (see [track](track.md)), queries
execute locally against index blocks fetched from the remote's raw storage
tier. Fetched artifacts are cached on disk, keyed by remote, under the OS
cache directory (`<cache-dir>/fluree/peer-cache/<remote>/`). Everything in
the cache is content-addressed and immutable — entries never go stale and
clearing is always safe (artifacts re-fetch on demand).

## Subcommands

### fluree cache status

Show per-remote disk usage.

```bash
fluree cache status
```

```
Peer cache: /Users/me/Library/Caches/fluree/peer-cache
origin 412.3 MiB
analytics 88.1 MiB
total 500.4 MiB
```

### fluree cache clear

Delete cached artifacts — everything, or one remote's with `--remote`.

```bash
fluree cache clear # all remotes
fluree cache clear --remote origin # one remote
```

## See Also

- [track](track.md) - Track a remote ledger (peer mode)
- [remote](remote.md) - Manage remotes and list accessible ledgers
20 changes: 20 additions & 0 deletions docs/cli/remote.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,26 @@ Remote:
Auth: token configured
```

### fluree remote ledgers

List the ledgers your token can access on a remote, with the serving tiers
each offers (`query` = the remote executes queries; `blocks` = raw index
content is served for peer-mode local execution). Requires a bearer token
with storage scope; the listing is the remote's auth-filtered catalog.

```bash
fluree remote ledgers [NAME] # NAME defaults to the only configured remote
```

```
LEDGER COMMIT T INDEX T SERVING
inventory:main 42 40 query+blocks
orders:main 17 17 query
```

Track one for peer-mode local querying with
`fluree track add <ledger> --remote <name> --mode peer` (see [track](track.md)).

## See Also

- [upstream](upstream.md) - Configure upstream tracking
Expand Down
Loading
Loading