go: bump franz-go to v1.21.2 and retire forked kgo-verifier pin#30655
Closed
travisdowns wants to merge 2 commits into
Closed
go: bump franz-go to v1.21.2 and retire forked kgo-verifier pin#30655travisdowns wants to merge 2 commits into
travisdowns wants to merge 2 commits into
Conversation
kgo-verifier was pinned to a forked franz-go pseudo-version (v1.20.6-0.20251204171952-b7b6b8e44d30) that carried the proposed AlwaysRetryEOF option from twmb/franz-go#1198 before it was released. That option shipped in franz-go v1.21.0, so we can drop the fork pin and move to a tagged release. Bump franz-go to v1.21.2 (+ kadm v1.18.0) across the go modules. Because tests/go/kgo-verifier and src/go/rpk share Bazel's single-version go.work, rpk moves to v1.21.2 as well; it builds and its kafka/topic tests pass against the new version. franz-go v1.21 requires Go >= 1.25, so the go directive is bumped where needed. Follow-up to CORE-14849.
The kgo-verifier producer/consumer workers already opt into AlwaysRetryEOF (via MakeKgoOpts) to avoid spurious non-retriable EOF failures when Redpanda is stopped/killed mid-handshake during chaos tests. Extend that workaround to the remaining test clients that build their own kgo options and were not covered: - kgo-repeater's NewAdmin() admin client - transform-verifier's common client builder - the transform-sdk integration test clients Follow-up to CORE-14849.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates franz-go-related Go dependencies across Redpanda’s Go test tooling and rpk, retiring the kgo-verifier fork now that AlwaysRetryEOF is available upstream.
Changes:
- Bumps franz-go/kadm/kmsg and related transitive dependencies across affected Go modules.
- Adds
kgo.AlwaysRetryEOF()to additional test Kafka client construction paths. - Raises Go directives for modules that now depend on franz-go requiring newer Go versions.
Reviewed changes
Copilot reviewed 8 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
tests/go/transform-verifier/go.mod |
Bumps franz-go, kadm, kmsg, and Go directive. |
tests/go/transform-verifier/go.sum |
Updates dependency checksums for transform-verifier. |
tests/go/transform-verifier/common/kgo.go |
Adds AlwaysRetryEOF to shared test client creation. |
tests/go/kgo-verifier/go.mod |
Moves off forked franz-go pseudo-version and updates related deps. |
tests/go/kgo-verifier/go.sum |
Updates kgo-verifier checksums. |
tests/go/kgo-verifier/cmd/kgo-repeater/main.go |
Adds AlwaysRetryEOF to repeater admin client. |
src/v/test_utils/go/go.mod |
Updates kmsg dependency. |
src/v/test_utils/go/go.sum |
Updates kmsg checksums. |
src/transform-sdk/tests/go.mod |
Bumps franz-go and related transitive dependencies for integration tests. |
src/transform-sdk/tests/go.sum |
Updates transform-sdk test dependency checksums. |
src/transform-sdk/tests/integration_test.go |
Adds AlwaysRetryEOF to transform-sdk test clients. |
src/go/rpk/go.mod |
Bumps rpk franz-go/kadm and compression dependency. |
src/go/rpk/go.sum |
Updates rpk dependency checksums. |
| github.com/testcontainers/testcontainers-go/modules/redpanda v0.34.0 | ||
| github.com/twmb/franz-go v1.19.4 | ||
| github.com/twmb/franz-go v1.21.2 | ||
| github.com/twmb/franz-go/pkg/kadm v1.16.1 |
Member
Author
|
Closing in favor of a re-scoped draft: dropping the transform-verifier and transform-sdk test-module changes (keeping this to the kgo-verifier franz-go bump + the rpk/test_utils versions it forces via the single-version go.work). Will link the replacement draft. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Bump
franz-goto v1.21.2 (andkadmto v1.18.0) across the repo's Go modules, and retire the forked pseudo-version pin (v1.20.6-0.20251204171952-b7b6b8e44d30) thattests/go/kgo-verifierwas carrying.That fork commit was the proposed
AlwaysRetryEOFoption from twmb/franz-go#1198, pulled in before it was released to work around CORE-14849 (test producers/consumers dying with a non-retriable EOF when Redpanda is stopped/killed mid-handshake during chaos tests). The option shipped in franz-go v1.21.0, so we can drop the fork and move to a tagged release.I also extend the
AlwaysRetryEOFopt-in to all test kafka clients, not just the kgo-verifier workers that already had it.Changes
franz-go v1.21.2,kadm v1.18.0) in:tests/go/kgo-verifier,src/go/rpk,src/v/test_utils/go(kmsg only),tests/go/transform-verifier,src/transform-sdk/tests.AlwaysRetryEOF()added to the test clients that built their own options and weren't covered:kgo-repeater'sNewAdmin(),transform-verifier'scommon.NewClient(), and thetransform-sdkintegration test clients. (kgo-verifier workers already opt in viaMakeKgoOpts.)godirective bumped to1.25.0where needed (franz-go v1.21 requires Go >= 1.25).Bazel / go.mod consistency
tests/go/kgo-verifierandsrc/go/rpkshare Bazel's single-versiongo.work, so this also moves rpk's Bazel build to franz-go v1.21.2. Verified:bazel build //src/go/rpk/cmd/rpk //tests/go/kgo-verifier/cmd/{kgo-verifier,kgo-repeater}- OKgo build ./...in each module - OKgo test ./pkg/kafka/... ./pkg/cli/topic/...in rpk - OKbazel mod tidy- no MODULE.bazel changes neededFollow-up
The vtools
qa/image/packer/remote-files/tests/go/kgo-verifiermirror (separate repo) will need the same bump to stay in sync.Backports Required
Release Notes