Skip to content

Commit dee854d

Browse files
committed
test(queue/sql): add comprehensive e2e integration tests
Why? The SQL queue implementation needs thorough integration testing against a real MySQL database to validate end-to-end functionality including: - Message publishing and consumption workflows - Partition management and isolation - Visibility timeout and retry mechanisms - DLQ (Dead Letter Queue) handling - Concurrency and crash recovery scenarios - Consumer group coordination and load balancing Without these tests, we cannot verify that the queue behaves correctly in production-like scenarios with a real database. What? - Add 15 comprehensive integration tests using testcontainers-go: * Basic operations: publish/subscribe, metadata preservation * Partition management: multiple partitions, isolation * Reliability: visibility timeout, retry, nack with delay * Correctness: idempotent publish, message ordering * Concurrency: concurrent publishers, concurrent subscribers * Crash recovery: lease expiration and message recovery * Consumer groups: multiple groups, multiple workers, load balancing * DLQ: max retries, DLQ metadata, failure tracking * Edge cases: late subscriber, empty topic, graceful shutdown - Add integration_tests/queue/README.md with: * Test coverage documentation * Prerequisites (Docker) * Running instructions (Bazel, go test, verbose output) * Test descriptions and what each validates - Refactor integration_tests/testutil for reusability: * Extract MySQL container utilities to testutil/mysql.go * Share MySQL setup across integration test suites * Add schema application helpers - Test infrastructure improvements: * Use testify/suite for setup/teardown lifecycle * Tag tests with "integration" for selective execution * Support for verbose test output (--test_arg=-test.v) * Proper container cleanup in all scenarios All 15 tests passing in ~2 minutes runtime. Co-Authored-By: Preetam Dwivedi <preetam@uber.com>
1 parent 462f8e4 commit dee854d

9 files changed

Lines changed: 1415 additions & 161 deletions

File tree

integration_tests/BUILD.bazel

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ load("@rules_go//go:def.bzl", "go_test")
55
go_test(
66
name = "integration_test",
77
srcs = [
8-
"mysql.go",
98
"servers.go",
109
"suite_test.go",
1110
],
@@ -18,6 +17,7 @@ go_test(
1817
],
1918
tags = ["integration"],
2019
deps = [
20+
"//integration_tests/testutil",
2121
"//gateway/protopb",
2222
"//orchestrator/protopb",
2323
"//speculator/protopb",

integration_tests/mysql.go

Lines changed: 0 additions & 123 deletions
This file was deleted.
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
load("@rules_go//go:def.bzl", "go_test")
2+
3+
go_test(
4+
name = "queue_test",
5+
srcs = ["queue_test.go"],
6+
data = [
7+
"//extensions/queue/sql/schema",
8+
],
9+
tags = ["integration"],
10+
deps = [
11+
"//entities/queue",
12+
"//extensions/queue",
13+
"//extensions/queue/sql",
14+
"//integration_tests/testutil",
15+
"@com_github_go_sql_driver_mysql//:mysql",
16+
"@com_github_stretchr_testify//assert",
17+
"@com_github_stretchr_testify//require",
18+
"@com_github_stretchr_testify//suite",
19+
"@com_github_testcontainers_testcontainers_go//:testcontainers-go",
20+
"@com_github_testcontainers_testcontainers_go//network",
21+
"@com_github_testcontainers_testcontainers_go_modules_mysql//:mysql",
22+
"@com_github_uber_go_tally_v4//:tally",
23+
"@org_uber_go_zap//zaptest",
24+
],
25+
)

0 commit comments

Comments
 (0)