Validator performance is causing missed-proofs to go undetected in the devnet.
Here's what happened.
- I ran my office NUC as a host for a few days. It picked up about 700 autoclient slots.
- Each slot has a proof-probability such that it will require proving on average 1x every 24h.
- On Friday, I simply pulled the plug on my NUC. Over the weekend, I expected the missed proofs to cause the NUC's slots to become freed and maybe repaired.
- On Monday, no slots were freed.
So I checked the validator logs and found that it takes our 1 devnet validator node aprox. 15 minutes to check all 3000 active devnet slots. Because the proof-period is 5 minutes, any missed proofs it finds can probably not be marked as missing simply because the time has already elapsed.
Improvements can be made to the validator to significantly speed up the checking of slots.
To continue testing the network's behavior, we will adjust the devnet to run 5 validator nodes instead of 1, using its configurations to segment the slot-space between them. This should result in slot-free events and repair in the network. Once validator improvements are implemented, we can scale this back down.
Validator performance is causing missed-proofs to go undetected in the devnet.
Here's what happened.
So I checked the validator logs and found that it takes our 1 devnet validator node aprox. 15 minutes to check all 3000 active devnet slots. Because the proof-period is 5 minutes, any missed proofs it finds can probably not be marked as missing simply because the time has already elapsed.
Improvements can be made to the validator to significantly speed up the checking of slots.
To continue testing the network's behavior, we will adjust the devnet to run 5 validator nodes instead of 1, using its configurations to segment the slot-space between them. This should result in slot-free events and repair in the network. Once validator improvements are implemented, we can scale this back down.