Skip to content

Change cluster client aggregation to test minimum clients#72

Open
jeremyprime wants to merge 1 commit intovalkey-io:mainfrom
jeremyprime:fix-flaky-client-agg-test
Open

Change cluster client aggregation to test minimum clients#72
jeremyprime wants to merge 1 commit intovalkey-io:mainfrom
jeremyprime:fix-flaky-client-agg-test

Conversation

@jeremyprime
Copy link
Copy Markdown
Collaborator

@jeremyprime jeremyprime commented Mar 12, 2026

Summary

This test frequently fails due to a race condition between measuring the cluster-wide client count and summing individual node counts as connections can be created/destroyed between the two measurements. Instead of verifying an exact count, verify the cluster-wide count is at least the minimum expected from individual nodes.

Testing

Ran CI tests a few times in a row without a single failure.

Signed-off-by: Jeremy Parr-Pearson <jeremy.parr-pearson@improving.com>
@jeremyprime jeremyprime marked this pull request as ready for review March 12, 2026 21:59
@jeremyprime jeremyprime requested a review from ikolomi March 12, 2026 21:59
@ikolomi
Copy link
Copy Markdown
Collaborator

ikolomi commented Mar 16, 2026

@jeremyprime

connections can be created/destroyed between the two measurements

how come?
this should not happen, we need to understand it because it might highlight a real problem

@jeremyprime
Copy link
Copy Markdown
Collaborator Author

@jeremyprime

connections can be created/destroyed between the two measurements

how come? this should not happen, we need to understand it because it might highlight a real problem

I believe this is due to the connection pooling. The connection pool maintains clients which have reconnects, keepalives, etc that continue to occur between the 2 getClientList() calls (plus the for loop is creating more connections on each getClientList() call) , causing the test to fail since the second count is larger than the first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants