Use MonitorUpdatingPersister for ChainMonitor#11
Conversation
Replace direct KVStore usage with MonitorUpdatingPersister to enable incremental channel monitor updates instead of rewriting the entire monitor on each update. This improves I/O efficiency by writing only differential updates (typically hundreds of bytes) rather than rewriting the full monitor (potentially tens of megabytes) for each state change. The existing behaviour tends to make channel updates very slow after a few thousand updates. The persister is configured with a maximum of 100 pending updates before consolidating to a full monitor write. This balances I/O bandwidth savings against storage complexity. Note: MonitorUpdatingPersister format is not backward-compatible with the standard persister due to the sentinel byte sequence prepended to monitors.
Use MonitorUpdatingPersister's read_all_channel_monitors_with_updates() method instead of the standard read_channel_monitors() function. The MonitorUpdatingPersister writes monitors with a special sentinel byte sequence (MONITOR_UPDATING_PERSISTER_PREPEND_SENTINEL) that makes them incompatible with the standard reading function. Only the persister's own reading method can properly deserialize monitors that may have unapplied updates. This fixes the "Failed to read ChannelMonitor" InvalidData error that occurs when attempting to load existing channel state.
There was a problem hiding this comment.
You might also need to call cleanup_stale_updates sometimes on startup to avoid accumulating abandoned updates. We're running with a maximum_pending_updates of 1000 in production now, but 100 sounds good to me as well.
Interestingly, I was looking for the previous lazy delete KVStore interface, but it looks like those changed to new async interfaces in LDK 0.2.
Is it also possible for CI to succeed here?
CI got borked a long time ago when we did the custom logging thing but I would like to get it working again. For now I rely on the Prober CI. I did just take a look since the custom logging stuff was reverted once it was available natively in LDK node. Looks like it is complaining about the intentional unreachable code should be easy to fix. I left it unreachable instead of deleting or commenting out to minimize conflicts |
2 participants
Replace direct KVStore usage with MonitorUpdatingPersister to enable incremental channel monitor updates instead of rewriting the entire monitor on each update. This improves I/O efficiency by writing only differential updates (typically hundreds of bytes) rather than rewriting the full monitor (potentially tens of megabytes) for each state change. The existing behaviour tends to make channel updates very slow after a few thousand updates.
The persister is configured with a maximum of 100 pending updates before consolidating to a full monitor write. This balances I/O bandwidth savings against storage complexity.
Note: MonitorUpdatingPersister format is not backward-compatible with the standard persister due to the sentinel byte sequence prepended to monitors.