Skip to content

Raft query peer event bus unification#1427

Open
zonotope wants to merge 7 commits into
mainfrom
fix/raft-query-peers
Open

Raft query peer event bus unification#1427
zonotope wants to merge 7 commits into
mainfrom
fix/raft-query-peers

Conversation

@zonotope

@zonotope zonotope commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

In raft mode, the StateMachineAdapter emits NameServiceEvents on RaftIntegration::event_bus, but the /v1/fluree/events SSE endpoint (and every other consumer that calls Fluree::event_bus()) subscribes on a different bus — the one Fluree allocates internally during build. Nothing bridged the two, so runtime raft commits never reached SSE subscribers. The bug hid because the endpoint's connection-time snapshot reads straight from the nameservice: data present at connect appeared, subsequent inserts did not. Query peers subscribed to a raft cluster therefore never heard about new commits.

Separately, Fluree's local cache event listener (which reconciles the LedgerManager on commit/index events and evicts on retract) was only spawned when background indexing was enabled. Query peers run without indexing, so they had no listener at all and depended on PeerSyncTask manually re-notifying the LedgerManager after every nameservice update.

Fix

FlureeBuilder gains with_event_bus(Arc<LedgerEventBus>): a caller-supplied bus is used instead of allocating a fresh one, uniformly across every build_* path (via a single resolve_event_bus helper). The raft server assembly threads RaftIntegration::event_bus through build_fluree_with_nameservice / build_default_fluree into the builder, so the state-machine adapter (publisher) and the SSE events endpoint, registry maintenance task, and cache listener (subscribers) all share one broadcast channel.

The local cache event listener is now spawned whenever a LedgerManager exists, independent of indexing mode. This gives query peers cache reconciliation driven by the events their NotifyingNameService already emits on successful CAS operations.

Consequent removals

  • The server assembly's explicit spawn_local_cache_event_listener call on the raft bus is gone — the builder-spawned listener now covers it (this also removes a duplicate reconcile task that previously ran on raft nodes with indexing enabled).
  • PeerSyncTask::refresh_cached_ledger and its manual LedgerManager notification are gone: fast_forward_commit dispatches through NotifyingNameService::compare_and_set_ref, which emits LedgerCommitPublished / LedgerIndexPublished on Updated, and the always-running listener reconciles from the just-updated local nameservice.
  • The explicit cache eviction in the peer retract path is gone: NotifyingNameService::retract emits LedgerRetracted, and the listener's ledger_manager.disconnect is exactly what Fluree::disconnect_ledger did.

Behavior change for embedders

The synchronous builders (build_memory(), build_with(), etc.) previously spawned no tasks when indexing was disabled and worked outside an async context. They now spawn the cache listener whenever ledger caching is enabled (the default), so all build_* methods must be called within a tokio runtime. The FlureeBuilder docs state this, and the affected unit tests were converted to #[tokio::test].

@zonotope zonotope requested review from aaj3f and bplatz July 3, 2026 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant