Skip to content

fix(config): default GoBinaryEnvVars to proxy.golang.org, not direct-only#2143

Closed
karlkfi wants to merge 1 commit into
gomods:mainfrom
karlkfi:proxy-first-default
Closed

fix(config): default GoBinaryEnvVars to proxy.golang.org, not direct-only#2143
karlkfi wants to merge 1 commit into
gomods:mainfrom
karlkfi:proxy-first-default

Conversation

@karlkfi

@karlkfi karlkfi commented Jun 30, 2026

Copy link
Copy Markdown

What

Change the default GoBinaryEnvVars from ["GOPROXY=direct"] to ["GOPROXY=https://proxy.golang.org,direct"].

Why

With the current default, Athens' internal go subprocess fetches each module from its origin VCS and rebuilds the module zip locally. When the Go toolchain bundled in the Athens image differs from the one that produced a module's notarized zip, the rebuilt zip hashes differently, and Athens rejects the valid module:

github.com/onsi/ginkgo/v2@v2.32.0: verifying module: checksum mismatch
	downloaded: h1:OPshXBha3Pavij2LzHWXX30meRBGIucLwh97bHLjLpk=
	sum.golang.org: h1:Hw7s2pVrQo/8Yz5N77qdnpHaoc+c6cC9WIV1Jce+J6E=
SECURITY ERROR
This download does NOT match the one reported by the checksum server.

The client just sees a 404. This is a recurring, hard-to-diagnose failure for ordinary public modules — e.g. a module declaring a go directive newer than the image's bundled Go. Prior reports: #1881, #1080, #1470.

Concretely reproduced on gomods/athens:v0.15.1 (bundles Go 1.20.14) fetching github.com/onsi/ginkgo/v2@v2.32.0 (declares go 1.25). Cross-checked that proxy.golang.org, sum.golang.org, a clean direct-VCS fetch with a current Go, and the module's go.sum all agree on h1:Hw7s2pVrQo... — only Athens' local rebuild diverged. The module is not tampered; the rebuild is.

The change

Defaulting to GOPROXY=https://proxy.golang.org,direct makes Athens fetch the immutable, notarized zip from the public proxy, whose hash matches sum.golang.org, so the checksum check passes. The ,direct fallback still resolves modules the proxy doesn't mirror. This matches Go's own default GOPROXY and eliminates the whole class of toolchain-skew checksum failures.

Operators who specifically need origin-only fetches (air-gapped / compliance) can still opt back in explicitly:

GoBinaryEnvVars = ["GOPROXY=direct"]

Testing

  • Updated config.dev.toml, the pkg/config/testdata/config.dev.toml fixture, and the TestParseExampleConfig assertion in lockstep so the defaultConfig() ⇄ example-config consistency test still holds.
  • go test ./pkg/config/... ./pkg/middleware/... ./pkg/module/... ./pkg/download/... all pass.

…only

The default GoBinaryEnvVars = ["GOPROXY=direct"] makes Athens' internal `go`
subprocess fetch each module from its origin VCS and rebuild the module zip
locally. When Athens' bundled Go toolchain differs from the one that produced
a module's notarized zip, the rebuilt zip hashes differently, and Athens
rejects the *valid* module with a Go "SECURITY ERROR" (checksum mismatch vs
sum.golang.org) and serves a 404. This is a recurring, hard-to-diagnose
failure for public modules — see gomods#1881, gomods#1080, and gomods#1470.

Default to GOPROXY=https://proxy.golang.org,direct instead. The public proxy
serves the immutable, notarized zip, whose hash matches sum.golang.org, so the
checksum check passes; the ",direct" fallback still resolves modules the proxy
doesn't mirror. This matches Go's own default GOPROXY and removes the whole
class of toolchain-skew checksum failures. Operators who need origin-only
fetches (air-gapped/compliance) can still set GoBinaryEnvVars =
["GOPROXY=direct"] explicitly.

Updated config.dev.toml, the testdata fixture, and the corresponding
assertions; pkg/config, pkg/middleware, pkg/module, and pkg/download tests
pass.

Signed-off-by: Karl Isenberg <karlkfi@gmail.com>
@matt0x6F

matt0x6F commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

I think this is probably the wrong way to solve this particular problem.

  1. a number of Athens' users can't access the public module proxy
  2. this is an abrupt breaking change
  3. the hash calculation is public and reproducible

We should probably seek to fix whatever is wrong with the hash calculation.

#1470 is where someone double published a tag. A lot of maintainers don't know that the proxy (and sumdb) see their version on release and that they can't simply delete published versions to issue a fix.

#1080 was a change in the way Go calculated checksums, not Athens, and would exist with your change.

#1881 is the exact same issue as #1470.

In the future we recommend starting with an issue so that we can guide you in the right direction. Second, I'd greatly appreciate doing some research — to include scanning maintainer comments of the issues you cited.

@karlkfi

karlkfi commented Jul 7, 2026

Copy link
Copy Markdown
Author

My mistake. I didn't realize , only tolerated HTTP 404 or 410. I guess we would need to use | to tolerate any error and fall through to direct in air-gapped or firewalled environments.

But I can also see how you might not want to add a failed GET to existing user's flows, even if it did fail gracefully.

That said, your responses to #1881 point at a re-pushed tag being the smoking gun, which doesn't seem to be the case in the ginkgo problem I encountered here. This seems like a different root cause entirely.

After further analysis, and referencing some more recent reports, here is a more in-depth bug report: #2145

@karlkfi
karlkfi deleted the proxy-first-default branch July 7, 2026 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants