Add deduplication or stable IDs for user-facing events

[joostjager]

Problem

Without stable event ids in LDK, ldk-node has no reliable upstream identity to persist with app-facing events. A replayed LDK event can therefore look like a fresh event to ldk-node, which can lead to duplicate app-facing events after restart.

Payment events usually carry enough payment-level identifiers for careful applications to dedupe the highest-risk side effects, but that is an event-specific workaround rather than a general event identity contract. For non-payment events, applications may create duplicate notifications, accounting entries, operational actions, or fulfillment work if they treat ldk-node events as exactly-once.

Upstream and Remaining ldk-node Work

The upstream rust-lightning side is tracked in https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4794. That issue covers the missing public logical identity for replayed lightning::events::Events.

If LDK exposes a stable event identity or idempotency token, ldk-node should persist it, or an app-event identity derived from it, with the queued app-facing event. On replay, ldk-node should then recognize that the corresponding app event was already queued or handled instead of appending a duplicate.

ldk-node should also decide whether to expose this identity to applications. If app-facing events remain at-least-once, exposing a stable event id or idempotency key would make that contract explicit and give applications a reliable dedupe key.

[joostjager]

Thinking this issue should probably also cover the persistence ordering inside EventQueue itself.

Currently EventQueue::add_event and EventQueue::event_handled mutate the in-memory queue before writing the serialized queue to the store. If the store write fails, the durable queue and in-memory queue can diverge: add_event can leave an event visible in the current process even though it was not durably queued, while event_handled can remove an event from memory even though it remains in the durable queue and may reappear after restart.

So besides stable IDs / dedupe for replayed LDK events, there is a simpler queue consistency question here too: should EventQueue write the prospective queue first and only mutate the live in-memory queue after persistence succeeds?

[tnull]

We discussed this at some point upstream, and I think if we want to do this we should add idempotency tokens in the LDK events first.

@joostjager Are you fine with moving the issue there?

[joostjager]

Make sense. Will move the issue there. The question about queue consistency in ldk-node (write before in-memory mutate) is one thing that remains?

[tnull]

Make sense. Will move the issue there. The question about queue consistency in ldk-node (write before in-memory mutate) is one thing that remains?

I guess we could switch to that model (recently did the same for DataStore). To do this efficiently we might need to switch the event queue mutex over to be a tokio::sync::Mutex, which in turn requires any methods accessing the queue to be async, would need to explore if that works out.

[joostjager]

I think today’s mechanism already has a similar ordering issue? It mutates the in-memory queue, takes a snapshot, but then releases the lock before writing. So there can be out-of-order writes?

So perhaps tokio mutex is needed either way?

[joostjager]

Created upstream issue https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4794 and updated body of this issue to describe the remaining work.