Fix/ci flakiness#4
Merged
Merged
Conversation
… CLA_SENT variable, narrow import_key error pattern
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(ci): eliminate intermittent test failures caused by blind sleeps and race conditions
Problem
The CI was failing on roughly every other run due to three distinct race conditions:
CLA signing — after signing the CLA on a fresh network deploy,
run_tests.shdid a blindsleep 10before running tests. On slow networks or freshly restarted nodes, the CLA transaction was not yet indexed when the first test transaction arrived, causingsignature verification failederrors.Key import errors silenced —
gnokey addwas called with> /dev/null 2>&1, masking any real failure (wrong mnemonic, corrupted keystore). Subsequent transaction failures were impossible to diagnose.Stress scripts race — all three sybil scripts (
sybil_chaos,sybil_precision,sybil_salted_chaos) deployed a counter realm with-broadcastthen immediately launched parallel transactions. Even though the deploy was committed, the RPC node had not yet indexed the package or updatedauth/accounts. Parallel transactions landed with a stale sequence →0 / 30 txs committed.Fix
run_tests.shsleep 10withwait_for_sequence_gtethat pollsauth/accountsuntil the runner's sequence reflects the committed CLA transactiongnokey addwithimport_keythat distinguishes "already exists" / real error / successsign_clanow setsCLA_SENT=1(variable) instead ofreturn 1to signal a tx was sentstress/common.sh(new file)wait_for_package PKG— pollsvm/qevaluntil the deployed package is queryable (max 30s)get_sequence ADDR— reads current committed sequence fromauth/accountswait_for_sequence_gte ADDR EXPECTED— polls until sequence is indexedsybil_chaos.sh,sybil_precision.sh,sybil_salted_chaos.shSEQ_BEFOREbeforeaddpkgwait_for_packagethenwait_for_sequence_gte SEQ_BEFORE+1before launching parallel transactions