Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions agent/flow-trace/00_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,3 +192,5 @@ _Found during source-code cross-referencing of these trace documents._
| 5 | **Active-job load balancing bug fixed** | Info | The Rust `NodeStateStore.available_tickets()` subtracts `active_jobs` from total tickets, reducing the chance of busy nodes being selected for new E3s. Previously, the `Sortition` actor's `Handler<InterfoldEvent>` was missing match arms for `E3Failed` and `E3StageChanged`, causing these events to fall to the default `_ => ()` — the typed handlers for decrementing jobs were dead code. This has been fixed: E3Failed and E3StageChanged are now routed to their handlers, and `finalized_committees` is cleaned up in `decrement_jobs_for_e3` to prevent unbounded memory growth. |
| 6 | **Committee member expulsion** | Info | `SlashingManager` can call `expelCommitteeMember()` mid-DKG. The `Sortition` actor enriches the raw `CommitteeMemberExpelled` event with the expelled member's `party_id` (resolved from its stored `Committee` list) and re-publishes it. `ThresholdKeyshare` then uses the enriched `party_id` to update its collectors, potentially completing DKG with fewer parties. `ThresholdKeyshare` itself does not hold committee state. |
| 7 | **ProofRequestActor failure bridge fixed** | Info | `ProofRequestActor` no longer leaves proof publication suppressed under log-only "will not be published" exits. `ComputeRequestError` and local proof-signing failures for DKG-path proofs (`C0` through `C5`) now emit `E3Failed { failed_at_stage: CommitteeFinalized, reason: DKGInvalidShares }`, while decryption-path proofs (`C6` and `C7`) emit `E3Failed { failed_at_stage: CiphertextReady, reason: DecryptionInvalidShares }`. |
| 8 | **Settlement receipts isolated** | Resolved | Durable EVM receipts such as `RewardCredited` and `RewardClaimed` are global audit/projection facts. The E3 router no longer sends them into a completed per-E3 context, so reward fan-out cannot reopen a finished E3 or produce false `AlreadyCompleted` failures. |
| 9 | **Replay-safe compute effects** | Resolved | `ComputeEffectGate` subscribes before EventStore replay, buffers `ComputeRequest`s while effects are disabled, deduplicates equivalent requests, prefers the newest hydrated retry, cancels terminal E3 work, and releases pending effects only after `EffectsEnabled`. This closes the mid-E3 compute-loss window without changing durable event order. |
7 changes: 7 additions & 0 deletions agent/flow-trace/02_TOKENS_AND_ACTIVATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -473,3 +473,10 @@ enforcement based on immutable policy curves. Key changes:
`BondingRegistry.getTicketBalanceAtBlock(node, c.requestBlock - 1)` — pass the value through
unchanged; the parameter is now a timepoint per EIP-6372 rather than a block number, which is
required for the tFOLD timestamp clock to be valid.

### Node-operator event projection

The EVM reader now emits typed `LicenseBondUpdated` and `CiphernodeDeregistrationRequested` events
alongside the existing ticket and activation events. The local dashboard rebuilds each chain's
registered/active node sets and the operator's ticket, license, and exit state from EventStore
history; it does not parse human-oriented CLI status output.
7 changes: 7 additions & 0 deletions agent/flow-trace/03_E3_REQUEST_AND_COMMITTEE.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,3 +414,10 @@ finalize the failure. Default `markFailedGracePeriod = 0` preserves the legacy p
`Committee.requestBlock` stores `block.timestamp` (EIP-6372 timestamp mode) so that `getPastVotes` /
`getTicketBalanceAtBlock` lookups against the `InterfoldTicketToken` resolve consistently across L1
and L2 clocks. The field name is preserved for storage / event ABI compatibility.

### Committee observability events

The EVM reader has typed coverage for `CommitteeFormationFailed`, `CommitteeActivationChanged`, and
`CommitteeViabilityUpdated` in addition to ticket submission, finalization, publication, and
expulsion. These facts are stored in the E3's chain aggregate and projected into the dashboard's
committee stage, including submitted/required thresholds and post-expulsion viability.
18 changes: 18 additions & 0 deletions agent/flow-trace/04_DKG_AND_COMPUTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1070,3 +1070,21 @@ ACTIVE AGGREGATOR collects PK_share₁ + PK_share₂ + PK_share₃
→ Produces aggregate_PK (public, published on-chain)
→ Anyone can encrypt, only committee can decrypt
```

## Durable flow tracing

The dashboard renders event ID, causation ID, origin ID, HLC timestamp, block watermark, aggregate,
and source exactly as the local EventStore recorded them. Observability does not change the
sequencing or gossip path. Local cause/effect chains are exact; received network events follow the
protocol's established receiver-local context semantics.

`InputPublished`, `RewardsDistributed`, `RewardCredited`, and `RewardClaimed` have typed EVM
translations. `CommitteePublished` and `PlaintextOutputPublished` are also translated from their
canonical on-chain logs. Every current interface signature is catalogued: logs without a
protocol-driving typed decoder become named, lossless `EvmLogObserved` facts, while a signature not
present in the running ABI catalog is exposed as `UnknownEvmLog` with raw topics/data.

During restart, `ComputeEffectGate` observes replay before compute workers are effects-enabled. It
buffers and deduplicates `ComputeRequest`s, prefers the newest regenerated request, cancels terminal
E3 work, and releases pending jobs only after `EffectsEnabled`. The gate changes effect timing, not
durable event order or audit state.
4 changes: 4 additions & 0 deletions agent/flow-trace/05_FAILURE_REFUND_SLASHING.md
Original file line number Diff line number Diff line change
Expand Up @@ -1149,6 +1149,10 @@ Applied audit findings: **C-05, H-05, H-06, H-07, H-09, H-10, H-24, M-14, M-15,
- `SlashProposed` and `SlashExecuted` carry a `Lane lane` field (`LaneA = 0`, `LaneB = 1`) so
off-chain indexers can disambiguate the two paths without re-deriving from policy bits.

The Rust minimal ABI matches the current `SlashExecuted` signature, including `bool executed` and
`Lane lane` (`uint8` at the ABI boundary). The typed actor event keeps the fields required for
expulsion handling; the full contract log is also retained by the raw EVM observability fallback.

### Upgrade posture

- `SlashingManager` is **non-upgradeable** by design (transparent proxy removed). Migrations require
Expand Down
14 changes: 14 additions & 0 deletions agent/flow-trace/06_DEACTIVATION_AND_COMPLETION.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,11 +220,20 @@ On restart:
│ 5. Historical libp2p sync retries failed aggregate fetches after reconnects
│ and also on bounded retry intervals even without a new connection event
│ 6. Sort & publish merged events by HLC timestamp
│ → ComputeEffectGate has already subscribed and buffers ComputeRequest
│ effects, deduplicating semantic retries while replay is in progress
│ 7. Enable effects (writers may submit only after this point)
│ → Gate cancels work for terminal E3s and releases only the newest
│ pending request for each in-flight semantic compute operation
│ 8. SyncEnded → live operations begin
└─ Node resumes from where it left off
```

Post-completion EVM receipts (`RewardsDistributed`, `RewardCredited`, `RewardClaimed`, and related
settlement observations) remain in EventStore for auditing and operator projections. The router does
not deliver them to a completed per-E3 context because they report settlement; they do not resume
protocol execution.

---

## Rust-Side: E3 Lifecycle Coordinator (durable stage tracking)
Expand Down Expand Up @@ -268,6 +277,11 @@ The coordinator is safe by construction during EventStore replay: observing a re
event simply re-derives the same monotonic stage, so the restored map is identical whether built
live or from replay.

The node-operator dashboard uses the same replay property. It pages every configured EventStore
aggregate and incrementally derives E3 stages, committees, tickets, failures, and rewards. The
projection is disposable and is rebuilt on restart; EventStore remains the only durable protocol
history.

---

## Exit Queue Timing
Expand Down
1 change: 1 addition & 0 deletions crates/ciphernode-builder/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ e3-evm.workspace = true
e3-fhe.workspace = true
e3-fhe-params.workspace = true
e3-keyshare.workspace = true
e3-logger.workspace = true
e3-multithread.workspace = true
e3-net.workspace = true
e3-slashing.workspace = true
Expand Down
25 changes: 24 additions & 1 deletion crates/ciphernode-builder/src/ciphernode.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ use actix::Addr;
use anyhow::Result;
use e3_data::{DataStore, InMemStore, StoreAddr};
use e3_events::{BusHandle, HistoryCollector, InterfoldEvent};
use e3_net::NetChannelBridge;
use e3_net::{NetChannelBridge, NetworkStatus};
use libp2p::PeerId;

use crate::global_eventstore_cache::EventStoreReader;

/// The kind of network interface backing a ciphernode.
#[derive(Debug, Clone)]
pub enum NetInterfaceKind {
Expand Down Expand Up @@ -47,6 +49,9 @@ pub struct CiphernodeHandle {
pub errors: Option<Addr<HistoryCollector<InterfoldEvent>>>,
pub peer_id: PeerId,
pub net_interface: NetInterfaceKind,
pub network_status: NetworkStatus,
pub eventstore: EventStoreReader,
pub aggregate_ids: Vec<usize>,
}

impl PartialEq for CiphernodeHandle {
Expand All @@ -66,6 +71,9 @@ impl CiphernodeHandle {
errors: Option<Addr<HistoryCollector<InterfoldEvent>>>,
peer_id: PeerId,
net_interface: NetInterfaceKind,
network_status: NetworkStatus,
eventstore: EventStoreReader,
aggregate_ids: Vec<usize>,
) -> Self {
Self {
address,
Expand All @@ -75,6 +83,9 @@ impl CiphernodeHandle {
errors,
peer_id,
net_interface,
network_status,
eventstore,
aggregate_ids,
}
}

Expand All @@ -94,6 +105,18 @@ impl CiphernodeHandle {
self.address.clone()
}

pub fn network_status(&self) -> NetworkStatus {
self.network_status.clone()
}

pub fn eventstore(&self) -> EventStoreReader {
self.eventstore.clone()
}

pub fn aggregate_ids(&self) -> &[usize] {
&self.aggregate_ids
}

pub fn store(&self) -> &DataStore {
&self.store
}
Expand Down
18 changes: 18 additions & 0 deletions crates/ciphernode-builder/src/ciphernode_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ use e3_evm::{
};
use e3_fhe::ext::FheExtension;
use e3_keyshare::ext::ThresholdKeyshareExtension;
use e3_logger::attach_protocol_logger;
use e3_multithread::{Multithread, MultithreadReport, TaskPool};
use e3_net::{
create_channel_bridge, setup_libp2p_keypair, setup_net, setup_net_interface,
Expand Down Expand Up @@ -72,6 +73,7 @@ pub struct CiphernodeBuilder {
in_mem_store: Option<Addr<InMemStore>>,
keyshare: Option<KeyshareKind>,
logging: bool,
name: Option<String>,
multithread_cache: Option<Addr<Multithread>>,
multithread_concurrent_jobs: Option<usize>,
multithread_report: Option<Addr<MultithreadReport>>,
Expand Down Expand Up @@ -136,6 +138,7 @@ impl CiphernodeBuilder {
in_mem_store: None,
keyshare: None,
logging: false,
name: None,
multithread_cache: None,
multithread_concurrent_jobs: None,
multithread_report: None,
Expand Down Expand Up @@ -171,6 +174,12 @@ impl CiphernodeBuilder {
self
}

/// Set the node name for dashboard display and log attribution.
pub fn with_name(mut self, name: &str) -> Self {
self.name = Some(name.to_string());
self
}

/// Use the TrBFV feature
pub fn with_trbfv(mut self) -> Self {
self.keyshare = Some(KeyshareKind::Threshold);
Expand Down Expand Up @@ -497,6 +506,11 @@ impl CiphernodeBuilder {
let addr = provider_cache.ensure_signer().await?.address().to_string();
let bus = event_system.handle()?.enable(&addr);

if self.logging {
let logger_name = self.name.as_deref().unwrap_or("ciphernode");
attach_protocol_logger(logger_name, &bus);
}

// Setup sortition
let (sortition, ciphernode_selector) =
self.setup_sortition(&bus, &repositories, &addr).await?;
Expand Down Expand Up @@ -531,6 +545,7 @@ impl CiphernodeBuilder {
// Setup networking
let topic = "interfold-gossip";
let (peer_id, interface, net_kind) = self.setup_networking(&store, topic).await?;
let network_status = interface.status();
setup_net(topic, bus.clone(), eventstore.ts(), interface)?;

// Run the sync routine
Expand All @@ -551,6 +566,9 @@ impl CiphernodeBuilder {
errors,
peer_id,
net_kind,
network_status,
eventstore,
aggregate_config.indexed_ids(),
))
}

Expand Down
2 changes: 1 addition & 1 deletion crates/ciphernode-builder/src/global_eventstore_cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use std::sync::OnceLock;
use actix::Recipient;
use e3_events::{EventStoreQueryBy, SeqAgg, TsAgg};

#[derive(Clone)]
#[derive(Clone, Debug)]
pub struct EventStoreReader {
query_by_seq: Recipient<EventStoreQueryBy<SeqAgg>>,
query_by_ts: Recipient<EventStoreQueryBy<TsAgg>>,
Expand Down
3 changes: 2 additions & 1 deletion crates/cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,18 @@ e3-config = { workspace = true }
e3-console = { workspace = true }
e3-ciphernode-builder = { workspace = true }
e3-crypto = { workspace = true }
e3-dashboard = { workspace = true }
e3-entrypoint = { workspace = true }
e3-events = { workspace = true }
e3-evm = { workspace = true }
e3-logger = { workspace = true }
e3-trbfv = { workspace = true }
e3-fhe-params = { workspace = true }
e3-zk-helpers = { workspace = true }
chrono = { workspace = true }
e3-init = { workspace = true }
e3-support-scripts = { workspace = true }
e3-daemon-server = { workspace = true }
e3-dashboard = { workspace = true }
e3-utils = { workspace = true }
e3-zk-prover = { workspace = true }
hex = { workspace = true }
Expand Down
13 changes: 11 additions & 2 deletions crates/cli/src/faucet.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,19 @@ use e3_evm::{
mod faucet_contract {
use super::sol;

// Declared inline rather than read from the Hardhat artifact JSON: test
// contract artifacts are not committed, so the JSON path fails to resolve
// on a clean checkout / CI.
sol!(
#[sol(rpc)]
FaucetContract,
"../../packages/interfold-contracts/artifacts/contracts/test/Faucet.sol/Faucet.json"
#[allow(clippy::too_many_arguments)]
interface FaucetContract {
function faucet() external;
function fold() external view returns (address);
function feeToken() external view returns (address);
function AMOUNT_FOLD() external view returns (uint256);
function AMOUNT_FEE_TOKEN() external view returns (uint256);
}
);
}

Expand Down
Loading
Loading