Skip to content

feat: event-time commits with @recorded: audit-axis time travel#1429

Open
bplatz wants to merge 1 commit into
feature/remote-mountsfrom
feature/event-time-commits
Open

feat: event-time commits with @recorded: audit-axis time travel#1429
bplatz wants to merge 1 commit into
feature/remote-mountsfrom
feature/event-time-commits

Conversation

@bplatz

@bplatz bplatz commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a second, independent time axis to commits so ledgers can support audit-axis time travel without disturbing existing event-time semantics.

Commits now distinguish two times:

  • Event time (Commit.time, existing field) — user-suppliable per transaction via opts.eventTime / CommitOpts::with_timestamp. @iso: time travel resolves against it, so historical data replayed as ordinary transactions gets real wall-clock time travel. Enforced monotonically non-decreasing along the chain and rejected beyond now + 5min skew (an immutable future stamp would permanently wedge the timeline). The default stamp is clamped to the head's event time, fixing pre-existing silent @iso: mis-resolution when the system clock stepped backwards between commits.

  • Recorded time (db:receivedAt, new system txn-meta entry) — wall-clock time the commit was recorded. Emitted from the first caller-supplied event time onward (sticky dual-stamp mode), so plain ledgers stay byte-identical. The new @recorded: selector (and TimeSpec::AtRecorded) time-travels the audit axis, falling back to the event axis for history before the flip point.

Implementation notes

  • HeadTemporal (event ms + receivedAt ms) rides LedgerState, captured during the load-time novelty walk and after each commit; resolved lazily via ensure_head_temporal (one storage read, write path only) using the branch-aware store — a flat store 404s on branched ledgers whose head commit lives in an ancestor namespace.
  • receivedAt reuses the txn_meta envelope channel (no wire-format change; old readers decode new commits).
  • The indexer resolver's reserved-namespace debug_assert now exempts system-injected entries; it already mis-fired on f:identity in debug builds.
  • apply_single_commit gains soft guards mirroring the build-path validation (event-time monotonicity + dual-stamp continuity).
  • fluree-memory git replay now backdates the schema transaction to the first git commit and clamps non-monotonic git author dates.
  • No format changes anywhere: no lower bound on event time for a fresh ledger (pre-1970 works), no migration, no cost to ledgers that never use the feature.

Testing

  • fluree-db-api/tests/it_event_time.rs (new) plus history-query coverage.
  • cargo check --workspace --all-features --all-targets — clean.

Notes

Stacked on feature/remote-mounts (#1428); review/merge that first. Docs updated: docs/concepts/time-travel.md, docs/api/endpoints.md.

@bplatz bplatz requested review from aaj3f and zonotope July 3, 2026 22:15
Commits now distinguish two time axes:

- Event time (Commit.time, existing field): user-suppliable per
  transaction via opts.eventTime / CommitOpts::with_timestamp. @iso:
  time travel resolves against it, so historical data replayed as
  ordinary transactions gets real wall-clock time travel. Enforced
  monotonically non-decreasing along the chain and rejected beyond
  now + 5min skew (an immutable future stamp would permanently wedge
  the timeline). The default stamp is clamped to the head's event time,
  fixing pre-existing silent @iso: mis-resolution when the system clock
  stepped backwards between commits.

- Recorded time (db:receivedAt, new system txn-meta entry): wall-clock
  time the commit was recorded. Emitted from the first caller-supplied
  event time onward (sticky dual-stamp mode), so plain ledgers stay
  byte-identical. The new @recorded: selector (and TimeSpec::AtRecorded)
  time-travels the audit axis, falling back to the event axis for
  history before the flip point.

Implementation notes:

- HeadTemporal (event ms + receivedAt ms) rides LedgerState, captured
  during the load-time novelty walk and after each commit; resolved
  lazily via ensure_head_temporal (one storage read, write path only)
  using the branch-aware store - a flat store 404s on branched ledgers
  whose head commit lives in an ancestor namespace.
- receivedAt reuses the txn_meta envelope channel (no wire-format
  change; old readers decode new commits). The indexer resolver's
  reserved-namespace debug_assert now exempts system-injected entries;
  it already mis-fired on f:identity in debug builds.
- apply_single_commit gains soft guards mirroring the build-path
  validation (event-time monotonicity + dual-stamp continuity).
- fluree-memory git replay now backdates the schema transaction to the
  first git commit and clamps non-monotonic git author dates.
- No format changes anywhere: no lower bound on event time for a fresh
  ledger (pre-1970 works), no migration, no cost to ledgers that never
  use the feature.
@bplatz bplatz force-pushed the feature/remote-mounts branch from 078af0b to 929b024 Compare July 4, 2026 14:07
@bplatz bplatz force-pushed the feature/event-time-commits branch from defbd6b to 4d4430c Compare July 4, 2026 14:07

@aaj3f aaj3f left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really neat! It's something we've always dismissed as separate from our time-axis concerns but I quite like this

Comment on lines +761 to +783
// Guard: event time must be monotonically non-decreasing along the
// chain. `datetime_to_t` (`@iso:` resolution) relies on this ordering;
// a violation would silently mis-resolve wall-clock time travel.
// Soft guard: only enforced when both sides are known — legacy commits
// without parseable times fall through rather than wedging the ledger.
let commit_temporal = HeadTemporal::from_commit(&commit);
if let (Some(prev), Some(new)) = (self.head_temporal, commit_temporal) {
if new.event_time_ms < prev.event_time_ms {
return Err(LedgerError::InvalidData(format!(
"Cannot apply commit at t={commit_t}: event time {} is earlier than \
the current head's event time {} (event time must be monotonically \
non-decreasing)",
new.event_time_ms, prev.event_time_ms
)));
}
if prev.dual_stamp() && !new.dual_stamp() {
return Err(LedgerError::InvalidData(format!(
"Cannot apply commit at t={commit_t}: ledger is in dual-stamp mode \
(head commit carries db:receivedAt) but this commit does not; \
@recorded: resolution requires every post-flip commit to dual-stamp"
)));
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The apply_single_commit event-time monotonicity guard hard-errors on the replay/catch-up path, where it is applied to commits that are already durable in the chain. The comment calls this a "soft guard … rather than wedging the ledger," but it only falls through for unparseable times — a legacy commit with a parseable but non-monotonic event time (real NTP step-back, or leadership handoff between raft nodes with skewed clocks — exactly the pre-existing scenario the PR description says exists) will return Err. apply_single_commit is the incremental catch-up path (ledger_manager.rs:1971), so a pre-PR ledger whose chain has any backwards clock step between two committed events will now fail incremental catch-up on upgrade. (A full reload via load_novelty/ bulk_apply_commits has no such guard, so behavior is also inconsistent between the two load paths, and the failure may repeat until a reload is forced.) The build path (resolve_commit_times) is the correct place to reject a new user commit; the replay path should tolerate history it can no longer change. Suggest warn-and-continue on apply:

let commit_temporal = HeadTemporal::from_commit(&commit);
if let (Some(prev), Some(new)) = (self.head_temporal, commit_temporal) {
    if new.event_time_ms < prev.event_time_ms {
        // Already-durable history: a pre-existing non-monotonic chain
        // (clock skew / pre-PR commits) must still load. The build path
        // rejects *new* violations; here we only surface the anomaly.
        tracing::warn!(
            commit_t, new = new.event_time_ms, prev = prev.event_time_ms,
            "applying commit whose event time predates the head; @iso: \
             resolution may be approximate across this point"
        );
    }
    if prev.dual_stamp() && !new.dual_stamp() {
        tracing::warn!(commit_t, "post-flip commit missing db:receivedAt");
    }
}

(If you want to keep the dual-stamp arm strict, note legacy ledgers can never be dual-stamped, so it only bites forward-built data and is lower risk than the monotonicity arm.)

Comment on lines +506 to +519
let received_at_ms = if dual_stamp {
let ms = match received {
Some(ts) => iso_to_epoch_ms_opt(&ts).ok_or_else(|| {
TransactError::InvalidEventTime(format!(
"receivedAt '{ts}' is not a valid RFC 3339 timestamp"
))
})?,
None => now_ms,
};
let prev = head.and_then(|h| h.received_time_ms);
Some(prev.map_or(ms, |p| ms.max(p)))
} else {
None
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The audit axis has no future-skew guard, unlike the event axis. resolve_commit_times validates that a caller-supplied event time is <= now + MAX_EVENT_TIME_FUTURE_SKEW_MS, but a caller-supplied received_at (via CommitOpts::with_received_at, reachable from the Rust API / CLI replay hooks) is accepted with any value. The PR's own argument — "commits are immutable and [the axis] is monotonic, so a future stamp would permanently pin the ledger's timeline ahead of reality" — applies verbatim to db:receivedAt. The HTTP route is safe (it always passes Utc::now()), so this is defense-in-depth for the programmatic path. Suggest applying the same future-bound check to the resolved received_at_ms:

let received_at_ms = if dual_stamp {
    let ms = match received { /* … */ };
    if ms > now_ms + MAX_EVENT_TIME_FUTURE_SKEW_MS {
        return Err(TransactError::InvalidEventTime(format!(
            "receivedAt is in the future; the audit axis is immutable and monotonic"
        )));
    }
    let prev = head.and_then(|h| h.received_time_ms);
    Some(prev.map_or(ms, |p| ms.max(p)))
} else { None };

Comment on lines +222 to +229
let earliest_flakes =
probe_timestamp_axis(snapshot, overlay, recv_predicate.clone(), current_t, None).await?;

let Some(earliest) = earliest_flakes.first() else {
// Never dual-stamped: the recorded axis is identical to event time.
tracing::debug!("recorded_to_t: no db:receivedAt flakes; falling back to event axis");
return datetime_to_t(snapshot, overlay, target_epoch_ms, current_t).await;
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@recorded: on a ledger that never dual-stamped does three single-flake POST probes (the empty receivedAt earliest-probe here, then datetime_to_t's two event-axis probes) versus @iso:'s two. Each probe is an O(log n) indexed single-flake read, so the absolute cost is small, but since the docs promise @recorded: is "identical to @iso: cost," it's worth a note. No change required; flag only if @recorded: is expected on hot query paths over plain ledgers.

Comment on lines +136 to +140
Once a ledger uses a caller-supplied event time, each commit records a second,
system-controlled timestamp: **`db:receivedAt`** — the wall-clock time the
commit was actually recorded. This begins at the first backdated commit and
continues on every commit thereafter (sticky), so the audit trail has no gaps.
Ledgers that never supply event times carry no extra metadata at all.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Once a ledger uses a caller-supplied event time, each commit records a second … db:receivedAt" is accurate for the HTTP path (the transact.rs route pairs eventTime with with_received_at(now)), but not for the Rust API: CommitOpts::with_timestamp alone does not flip dual-stamp mode (resolve_commit_times gates it on received.is_some() || head.dual_stamp()). fluree-memory git replay backdates via with_timestamp only, so those ledgers are event-axis-only and never carry receivedAt. Consider clarifying that the audit axis is enabled by the server pairing the two stamps (or by with_received_at directly), not by any event-time use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants