Skip to content

fix (test): use TCP healthcheck for MySQL containers to prevent flaky startup#64

Merged
behinddwalls merged 3 commits into
mainfrom
preetam/fix-orch
Feb 24, 2026
Merged

fix (test): use TCP healthcheck for MySQL containers to prevent flaky startup#64
behinddwalls merged 3 commits into
mainfrom
preetam/fix-orch

Conversation

@behinddwalls
Copy link
Copy Markdown
Collaborator

Summary

  • Switch MySQL healthchecks from localhost (Unix socket) to 127.0.0.1 (TCP) across all 6 docker-compose files
  • Remove retry loop from ConnectMySQLService since TCP healthcheck guarantees readiness before Up() returns

MySQL treats localhost as a Unix socket connection, which can be ready before the TCP listener. Dependent services (orchestrator, gateway) connecting over TCP would intermittently get connection refused — causing flaky integration tests and CI failures.

Test plan

  • Reproduced flakiness locally (orchestrator failed 2/3 runs, gateway failed 3/3 runs)
  • Verified fix: 4 consecutive uncached orchestrator runs pass, 3 consecutive gateway runs pass
  • All 5 integration tests pass
  • E2E test passes

MySQL treats "localhost" as a Unix socket connection, which can be ready
before the TCP listener. Dependent services connecting over TCP would
intermittently get "connection refused". Switching to 127.0.0.1 forces
TCP-based healthchecks, matching the actual connection path. Also removes
retry loop from ConnectMySQLService since the healthcheck now guarantees
TCP readiness.
@behinddwalls behinddwalls requested review from a team and sbalabanov as code owners February 24, 2026 20:42
@behinddwalls behinddwalls changed the title fix: use TCP healthcheck for MySQL containers to prevent flaky startup fix (test): use TCP healthcheck for MySQL containers to prevent flaky startup Feb 24, 2026
@behinddwalls behinddwalls merged commit 0e2d740 into main Feb 24, 2026
8 checks passed
@behinddwalls behinddwalls deleted the preetam/fix-orch branch February 24, 2026 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants