Skip to content

Fix/ci flakiness#4

Merged
louis14448 merged 13 commits into
mainfrom
fix/ci-flakiness
Jun 2, 2026
Merged

Fix/ci flakiness#4
louis14448 merged 13 commits into
mainfrom
fix/ci-flakiness

Conversation

@louis14448

Copy link
Copy Markdown
Contributor

fix(ci): eliminate intermittent test failures caused by blind sleeps and race conditions

Problem

The CI was failing on roughly every other run due to three distinct race conditions:

  1. CLA signing — after signing the CLA on a fresh network deploy, run_tests.sh did a blind sleep 10 before running tests. On slow networks or freshly restarted nodes, the CLA transaction was not yet indexed when the first test transaction arrived, causing signature verification failed errors.

  2. Key import errors silencedgnokey add was called with > /dev/null 2>&1, masking any real failure (wrong mnemonic, corrupted keystore). Subsequent transaction failures were impossible to diagnose.

  3. Stress scripts race — all three sybil scripts (sybil_chaos, sybil_precision, sybil_salted_chaos) deployed a counter realm with -broadcast then immediately launched parallel transactions. Even though the deploy was committed, the RPC node had not yet indexed the package or updated auth/accounts. Parallel transactions landed with a stale sequence → 0 / 30 txs committed.

Fix

run_tests.sh

  • Replace sleep 10 with wait_for_sequence_gte that polls auth/accounts until the runner's sequence reflects the committed CLA transaction
  • Replace silent gnokey add with import_key that distinguishes "already exists" / real error / success
  • sign_cla now sets CLA_SENT=1 (variable) instead of return 1 to signal a tx was sent

stress/common.sh (new file)

  • wait_for_package PKG — polls vm/qeval until the deployed package is queryable (max 30s)
  • get_sequence ADDR — reads current committed sequence from auth/accounts
  • wait_for_sequence_gte ADDR EXPECTED — polls until sequence is indexed

sybil_chaos.sh, sybil_precision.sh, sybil_salted_chaos.sh

  • Capture SEQ_BEFORE before addpkg
  • After deploy: wait_for_package then wait_for_sequence_gte SEQ_BEFORE+1 before launching parallel transactions

@louis14448
louis14448 merged commit ec29b03 into main Jun 2, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant