Skip to content

Commit e632592

Browse files
albertywuclaude
andcommitted
build(proto): make make proto fully hermetic
`make proto` previously used the host's protoc and plugins (Homebrew protoc, `go install ...@latest` for protoc-gen-go / protoc-gen-go-grpc / protoc-gen-yarpc-go), so generated stubs could drift between machines. Pin the whole toolchain so one command produces identical output anywhere. - Pin protoc via ./tool/protoc, a Bazelisk-style wrapper that downloads and SHA-256-verifies the version in .protocversion (29.3) and caches it. - Pin all three plugins via `tool` directives in go.mod at the versions that reproduce the committed output (protoc-gen-go v1.36.10, protoc-gen-go-grpc v1.5.1, protoc-gen-yarpc-go from yarpc v1.81.0); protoc-gen-go-grpc is a separate module, now added to the graph. - Add tool/protoc-gen-{go,go-grpc,yarpc-go} wrappers that run the pinned plugins via `go tool`, and pass them to protoc with explicit --plugin= flags so $PATH is never consulted. - Run a pinned goimports pass so `make proto` emits the committed (formatted) form in a single command. - Add a `check-proto` target wired into the CI `tidy` job so stale generated files fail the required-checks gate. - Drop the host protoc/plugin install steps from DEVELOPMENT.md. No change to the generated stubs: the committed yarpc output stays on the existing gogo codec. Switching to the protobuf v2 codec will be a separate PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent f17c76f commit e632592

12 files changed

Lines changed: 222 additions & 16 deletions

File tree

.github/workflows/ci.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ jobs:
3636
run: make lint
3737

3838
# ---------------------------------------------------------------------------
39-
# TIDY (module files + BUILD files in sync)
39+
# TIDY (module files + BUILD files + generated proto in sync)
4040
# ---------------------------------------------------------------------------
4141
tidy:
4242
name: Tidy
@@ -52,6 +52,9 @@ jobs:
5252
- name: Check BUILD files are up to date
5353
run: make check-gazelle
5454

55+
- name: Check generated proto files are up to date
56+
run: make check-proto
57+
5558
# ---------------------------------------------------------------------------
5659
# BUILD AND UNIT TESTS
5760
# ---------------------------------------------------------------------------

.protocversion

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
29.3

MODULE.bazel

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ use_repo(
4949
"com_github_uber_go_tally_v4",
5050
"in_gopkg_yaml_v3",
5151
"org_golang_google_grpc",
52+
"org_golang_google_grpc_cmd_protoc_gen_go_grpc",
5253
"org_golang_google_protobuf",
5354
"org_golang_x_oauth2",
5455
"org_uber_go_fx",

Makefile

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,16 @@
11
# Bazel wrapper
22
BAZEL = ./tool/bazel
33

4+
# protoc wrapper (hermetic; pinned by .protocversion)
5+
PROTOC = ./tool/protoc
6+
7+
# protoc plugins (hermetic; versions pinned by the `tool` directives in go.mod).
8+
# Passed explicitly so protoc never resolves a plugin from the host $PATH.
9+
PROTOC_PLUGINS = \
10+
--plugin=protoc-gen-go=$(CURDIR)/tool/protoc-gen-go \
11+
--plugin=protoc-gen-go-grpc=$(CURDIR)/tool/protoc-gen-go-grpc \
12+
--plugin=protoc-gen-yarpc-go=$(CURDIR)/tool/protoc-gen-yarpc-go
13+
414
# Docker Compose wrapper
515
COMPOSE = docker-compose
616

@@ -40,7 +50,7 @@ define assert_clean
4050
fi
4151
endef
4252

43-
.PHONY: build build-all-linux build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-gateway-linux build-stovepipe-orchestrator-linux check-gazelle check-mocks check-tidy clean clean-proto deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-fmt lint-license local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-init-stovepipe-queue-schema local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-gateway-start local-stovepipe-orchestrator-start local-stovepipe-start mocks proto query-deps query-targets run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe-gateway run-client-stovepipe-orchestrator run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
53+
.PHONY: build build-all-linux build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-gateway-linux build-stovepipe-orchestrator-linux check-gazelle check-mocks check-proto check-tidy clean clean-proto deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-fmt lint-license local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-init-stovepipe-queue-schema local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-gateway-start local-stovepipe-orchestrator-start local-stovepipe-start mocks proto query-deps query-targets run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe-gateway run-client-stovepipe-orchestrator run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
4454

4555

4656
build: ## Build all services and examples
@@ -94,6 +104,10 @@ check-mocks: mocks ## Check mock files are up to date
94104
$(call assert_clean,make mocks)
95105
@echo "Mock files are up to date."
96106

107+
check-proto: proto ## Check generated proto files are up to date
108+
$(call assert_clean,make proto)
109+
@echo "Proto files are up to date."
110+
97111
check-tidy: tidy ## Check that go.mod and MODULE.bazel are tidy
98112
$(call assert_clean,make tidy)
99113
@echo "Module files are up to date."
@@ -339,22 +353,26 @@ mocks: ## Generate mock files using mockgen
339353

340354
proto: ## Generate protobuf files from .proto definitions
341355
@echo "Generating protobuf files with protoc..."
342-
@protoc --go_out=submitqueue/gateway/protopb --go_opt=paths=source_relative \
356+
@$(PROTOC) $(PROTOC_PLUGINS) --go_out=submitqueue/gateway/protopb --go_opt=paths=source_relative \
343357
--go-grpc_out=submitqueue/gateway/protopb --go-grpc_opt=paths=source_relative \
344358
--yarpc-go_out=submitqueue/gateway/protopb --yarpc-go_opt=paths=source_relative \
345359
--proto_path=submitqueue/gateway/proto submitqueue/gateway/proto/gateway.proto
346-
@protoc --go_out=submitqueue/orchestrator/protopb --go_opt=paths=source_relative \
360+
@$(PROTOC) $(PROTOC_PLUGINS) --go_out=submitqueue/orchestrator/protopb --go_opt=paths=source_relative \
347361
--go-grpc_out=submitqueue/orchestrator/protopb --go-grpc_opt=paths=source_relative \
348362
--yarpc-go_out=submitqueue/orchestrator/protopb --yarpc-go_opt=paths=source_relative \
349363
--proto_path=submitqueue/orchestrator/proto submitqueue/orchestrator/proto/orchestrator.proto
350-
@protoc --go_out=stovepipe/gateway/protopb --go_opt=paths=source_relative \
364+
@$(PROTOC) $(PROTOC_PLUGINS) --go_out=stovepipe/gateway/protopb --go_opt=paths=source_relative \
351365
--go-grpc_out=stovepipe/gateway/protopb --go-grpc_opt=paths=source_relative \
352366
--yarpc-go_out=stovepipe/gateway/protopb --yarpc-go_opt=paths=source_relative \
353367
--proto_path=stovepipe/gateway/proto stovepipe/gateway/proto/gateway.proto
354-
@protoc --go_out=stovepipe/orchestrator/protopb --go_opt=paths=source_relative \
368+
@$(PROTOC) $(PROTOC_PLUGINS) --go_out=stovepipe/orchestrator/protopb --go_opt=paths=source_relative \
355369
--go-grpc_out=stovepipe/orchestrator/protopb --go-grpc_opt=paths=source_relative \
356370
--yarpc-go_out=stovepipe/orchestrator/protopb --yarpc-go_opt=paths=source_relative \
357371
--proto_path=stovepipe/orchestrator/proto stovepipe/orchestrator/proto/orchestrator.proto
372+
@echo "Formatting generated files with goimports..."
373+
@go run golang.org/x/tools/cmd/goimports@$(GOIMPORTS_VERSION) -w \
374+
submitqueue/gateway/protopb submitqueue/orchestrator/protopb \
375+
stovepipe/gateway/protopb stovepipe/orchestrator/protopb
358376
@echo "Protobuf files generated successfully!"
359377

360378
# Bazel query helpers

doc/howto/DEVELOPMENT.md

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Prerequisites
44

5-
- **Go 1.24+** — needed for `gopls`, `go mod`, and installing protoc plugins. Download from [go.dev/dl](https://go.dev/dl/). Note: Bazel manages its own Go toolchain for builds, but a local Go installation is required for editor tooling and dependency management.
5+
- **Go 1.24+** — needed for `gopls`, `go mod`, and running the hermetic protoc plugins (via `go tool`). Download from [go.dev/dl](https://go.dev/dl/). Note: Bazel manages its own Go toolchain for builds, but a local Go installation is required for editor tooling and dependency management.
66
- **Docker** and **Docker Compose** — for integration and e2e tests, and for running services locally.
77
- **direnv** (recommended) — automatically loads `.envrc` so you can use `bazel` directly instead of `./tool/bazel`.
88

@@ -85,15 +85,17 @@ GoLand works with Go modules automatically. Open the project root and GoLand wil
8585
## Optional Tools
8686

8787
```bash
88-
# macOS
89-
brew install protobuf grpcurl
90-
91-
# Go protoc plugins (only if modifying .proto files)
92-
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
93-
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
94-
go install go.uber.org/yarpc/encoding/protobuf/protoc-gen-yarpc-go@latest
88+
# macOS — grpcurl for poking at running services (optional)
89+
brew install grpcurl
9590
```
9691

92+
Proto generation is fully hermetic and needs no manual installs: `make proto`
93+
downloads a pinned `protoc` via `./tool/protoc` (see `.protocversion`) and runs
94+
the `protoc-gen-go`, `protoc-gen-go-grpc`, and `protoc-gen-yarpc-go` plugins
95+
at the versions pinned by the `tool` directives in `go.mod` (via `go tool`). A
96+
Go toolchain (and network access on the first run, to fetch protoc and the
97+
plugin modules) is the only requirement.
98+
9799
## Common Make Targets
98100

99101
| Target | Description |
@@ -138,8 +140,9 @@ See [TESTING.md](TESTING.md) for the full testing guide, including integration a
138140
## Troubleshooting
139141

140142
**Proto generation fails:**
141-
- Ensure all three protoc plugins are installed (see Optional Tools above)
142-
- Check that `protoc` is in your PATH: `which protoc`
143+
- `make proto` is hermetic — it needs no host `protoc` or plugins, only a Go toolchain and (on the first run) network access to download pinned `protoc` and the plugin modules.
144+
- To bump versions: edit `.protocversion` (and add the new platform checksums in `./tool/protoc`) for protoc, or `go get -tool <plugin>@<version>` followed by `make tidy` for a plugin.
145+
- Run `make check-proto` to confirm the committed generated files match a fresh `make proto`.
143146

144147
**Build fails after proto changes:**
145148
- Run `make proto` to regenerate proto files

go.mod

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,13 @@ require (
5656
golang.org/x/tools v0.41.0 // indirect
5757
golang.org/x/tools/go/expect v0.1.1-deprecated // indirect
5858
google.golang.org/genproto/googleapis/rpc v0.0.0-20241230172942-26aa7a208def // indirect
59+
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1 // indirect
5960
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
6061
honnef.co/go/tools v0.4.3 // indirect
6162
)
63+
64+
tool (
65+
go.uber.org/yarpc/encoding/protobuf/protoc-gen-yarpc-go
66+
google.golang.org/grpc/cmd/protoc-gen-go-grpc
67+
google.golang.org/protobuf/cmd/protoc-gen-go
68+
)

go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,8 @@ google.golang.org/genproto/googleapis/rpc v0.0.0-20241230172942-26aa7a208def/go.
296296
google.golang.org/grpc v1.12.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
297297
google.golang.org/grpc v1.68.1 h1:oI5oTa11+ng8r8XMMN7jAOmWfPZWbYpCFaMUTACxkM0=
298298
google.golang.org/grpc v1.68.1/go.mod h1:+q1XYFJjShcqn0QZHvCyeR4CXPA+llXIeUIfIe00waw=
299+
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1 h1:F29+wU6Ee6qgu9TddPgooOdaqsxTMunOoj8KA5yuS5A=
300+
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1/go.mod h1:5KF+wpkbTSbGcR9zteSqZV6fqFOWBl4Yde8En8MryZA=
299301
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
300302
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
301303
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=

tool/BUILD.bazel

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,8 @@
33

44
exports_files([
55
"bazel",
6+
"protoc",
7+
"protoc-gen-go",
8+
"protoc-gen-go-grpc",
9+
"protoc-gen-yarpc-go",
610
])

tool/protoc

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Copyright (c) 2025 Uber Technologies, Inc.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
# Hermetic protoc wrapper. Downloads and pins protoc to the version in
18+
# .protocversion, then execs it. This mirrors ./tool/bazel (Bazelisk) so that
19+
# `make proto` produces identical output regardless of any host/homebrew protoc.
20+
#
21+
# The binary is cached under ${XDG_CACHE_HOME:-$HOME/.cache}/submitqueue-protoc/
22+
# and verified against a pinned SHA-256 before use.
23+
24+
set -euo pipefail
25+
26+
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
27+
version="$(tr -d '[:space:]' <"${repo_root}/.protocversion")"
28+
29+
# Map the host OS/arch onto the protobuf release asset suffix.
30+
os="$(uname -s)"
31+
arch="$(uname -m)"
32+
case "${os}" in
33+
Darwin) os_part="osx" ;;
34+
Linux) os_part="linux" ;;
35+
*) echo "tool/protoc: unsupported OS: ${os}" >&2; exit 1 ;;
36+
esac
37+
case "${arch}" in
38+
arm64 | aarch64) arch_part="aarch_64" ;;
39+
x86_64 | amd64) arch_part="x86_64" ;;
40+
*) echo "tool/protoc: unsupported arch: ${arch}" >&2; exit 1 ;;
41+
esac
42+
plat="${os_part}-${arch_part}"
43+
44+
# Pinned SHA-256 of protoc-${version}-${plat}.zip. Add new entries when bumping
45+
# .protocversion (see the release page on github.com/protocolbuffers/protobuf).
46+
checksum=""
47+
case "${version}:${plat}" in
48+
"29.3:osx-aarch_64") checksum="2b8a3403cd097f95f3ba656e14b76c732b6b26d7f183330b11e36ef2bc028765" ;;
49+
"29.3:osx-x86_64") checksum="9a788036d8f9854f7b03c305df4777cf0e54e5b081e25bf15252da87e0e90875" ;;
50+
"29.3:linux-x86_64") checksum="3e866620c5be27664f3d2fa2d656b5f3e09b5152b42f1bedbf427b333e90021a" ;;
51+
"29.3:linux-aarch_64") checksum="6427349140e01f06e049e707a58709a4f221ae73ab9a0425bc4a00c8d0e1ab32" ;;
52+
esac
53+
54+
cache_root="${XDG_CACHE_HOME:-${HOME}/.cache}/submitqueue-protoc"
55+
install_dir="${cache_root}/${version}/${plat}"
56+
protoc_bin="${install_dir}/bin/protoc"
57+
58+
sha256() {
59+
if command -v sha256sum >/dev/null 2>&1; then
60+
sha256sum "$1" | awk '{print $1}'
61+
else
62+
shasum -a 256 "$1" | awk '{print $1}'
63+
fi
64+
}
65+
66+
if [[ ! -x "${protoc_bin}" ]]; then
67+
url="https://github.com/protocolbuffers/protobuf/releases/download/v${version}/protoc-${version}-${plat}.zip"
68+
tmp="$(mktemp -d)"
69+
trap 'rm -rf "${tmp}"' EXIT
70+
echo "tool/protoc: downloading protoc ${version} (${plat})..." >&2
71+
curl -fsSL -o "${tmp}/protoc.zip" "${url}"
72+
if [[ -n "${checksum}" ]]; then
73+
got="$(sha256 "${tmp}/protoc.zip")"
74+
if [[ "${got}" != "${checksum}" ]]; then
75+
echo "tool/protoc: checksum mismatch for protoc-${version}-${plat}.zip" >&2
76+
echo " expected ${checksum}" >&2
77+
echo " got ${got}" >&2
78+
exit 1
79+
fi
80+
else
81+
echo "tool/protoc: no pinned checksum for ${version}:${plat}; add one to tool/protoc" >&2
82+
echo " (downloaded sha256: $(sha256 "${tmp}/protoc.zip"))" >&2
83+
fi
84+
rm -rf "${install_dir}"
85+
mkdir -p "${install_dir}"
86+
unzip -q "${tmp}/protoc.zip" -d "${install_dir}"
87+
fi
88+
89+
exec "${protoc_bin}" "$@"

tool/protoc-gen-go

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Copyright (c) 2025 Uber Technologies, Inc.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
# Hermetic protoc-gen-go wrapper. Runs the version pinned by the `tool`
18+
# directive in go.mod via the Go toolchain, so `make proto` produces identical
19+
# output regardless of any host/`go install`ed protoc-gen-go. Mirrors
20+
# ./tool/protoc (which pins protoc itself). protoc invokes this with no args and
21+
# speaks the plugin protocol over stdin/stdout.
22+
23+
set -euo pipefail
24+
25+
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
26+
exec go -C "${repo_root}" tool protoc-gen-go "$@"

0 commit comments

Comments
 (0)