Server#tick_loop uses select with a timeout(50.milliseconds) arm to drive @node.tick. Crystal's select only fires the timeout case when no other case is ready — so under sustained inbox/actions load, tick is never called. raft uses tick for heartbeat sends (leader) and election timeouts (followers), so a busy node becomes less reactive the busier it gets.
Not a problem for low-frequency metadata Nodes — select is mostly idle, timeout fires reliably. Becomes a problem for high-throughput data-plane Nodes where every operation translates to an inbox message or propose.
Proposed solution
A dedicated tick-timer fiber that sends on a buffered(1) channel makes tick a first-class select case competing fairly with inbox/actions:
@tick_signal = Channel(Nil).new(1)
spawn { loop { sleep 50.ms; select when @tick_signal.send(nil); else end } }
# tick_loop's select replaces `when timeout(50.ms)` with `when @tick_signal.receive?`
The buffered(1) + select-else pattern coalesces missed ticks rather than backing them up.
The queue example uses the same select+timeout pattern and would benefit similarly.
Server#tick_loopusesselectwith atimeout(50.milliseconds)arm to drive@node.tick. Crystal'sselectonly fires the timeout case when no other case is ready — so under sustained inbox/actions load, tick is never called. raft uses tick for heartbeat sends (leader) and election timeouts (followers), so a busy node becomes less reactive the busier it gets.Not a problem for low-frequency metadata Nodes —
selectis mostly idle, timeout fires reliably. Becomes a problem for high-throughput data-plane Nodes where every operation translates to an inbox message or propose.Proposed solution
A dedicated tick-timer fiber that sends on a buffered(1) channel makes
ticka first-class select case competing fairly withinbox/actions:The buffered(1) +
select-else pattern coalesces missed ticks rather than backing them up.The queue example uses the same
select+timeoutpattern and would benefit similarly.