Skip to content

Allow the usage of mirror registries/repositories in all/most places#984

Open
mozesl-nokia wants to merge 9 commits into
kptdev:mainfrom
nokia:mirror-mirror
Open

Allow the usage of mirror registries/repositories in all/most places#984
mozesl-nokia wants to merge 9 commits into
kptdev:mainfrom
nokia:mirror-mirror

Conversation

@mozesl-nokia
Copy link
Copy Markdown
Contributor

@mozesl-nokia mozesl-nokia commented May 15, 2026

Title

Allow the usage of mirror registries/repositories in all/most places


Description

  • What changed: You can now pass even more environment variables to replace default registries/repositories:
    • Dockerfiles
    • Make targets
    • Bash scripts
    • DB unit test suite
  • Why it’s needed: So that you can work more easily behind (increasingly stricter) proxies
  • How it works: Define env vars before commands or use a .env file for convenience

Related Issue(s)

(I thought we opened (and fixed) an issue for this, but I cannot find it.)


Type of Change

  • Other: Development environment

Checklist

  • Code follows project style guidelines
  • Self-reviewed changes
  • Tests added/updated
  • Documentation added/updated
  • All tests and gating checks pass

Testing Instructions (Optional)

  1. make setup-dev-env
  2. make run-in-kind-db-cache
  3. Tried a few other make targets, like reload-server and reload-function-runner, but please try your favourite make targets and scripts too to make sure they all work.

Additional Notes (Optional)

  • Further improvements: Somehow add the GHCR mirror to the prefix list of the FunctionConfigs in the example deployment.
  • Review notes: Added a few comments like: "TODO: do we still use this?", I am pretty sure these are unused and should be removed. Please check. Also, I am open to changing the names of the env vars.

AI Disclosure

  • I have used AI in the creation of this PR.

I used Cursor CLI on Auto to generate the script to import .env vars in common.sh

Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds environment-variable-driven registry/mirror overrides for development builds, deployment scripts, kpt functions, and DB tests so contributors can work behind registry/proxy restrictions.

Changes:

  • Adds .env/environment support for GHCR, Docker Hub, and Maven mirror configuration.
  • Replaces hardcoded kpt function and Docker base image registries with configurable prefixes.
  • Extends DB tests and dev setup scripts to consume mirror-related environment variables.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
.env.template Documents mirror-related environment variables.
Makefile Exports registry mirror defaults and adds dump-env.
build/Dockerfile Makes Docker Hub base images configurable.
build/Dockerfile.apiserver Adds usage TODO comment.
build/Dockerfile.etcd Adds usage TODO comment.
build/Makefile Passes Docker Hub mirror build args.
controllers/Dockerfile Makes builder base image configurable.
controllers/Makefile Passes Docker Hub mirror build args.
func/Dockerfile Makes kpt function and base image registries configurable.
func/Dockerfile-wrapperserver Makes base images configurable.
func/Makefile Passes mirror-related build args.
go.mod Adds godotenv dependency.
go.sum Adds checksums for godotenv.
make/deploy.mk Adjusts setup-dev-env help placement.
pkg/cache/dbcache/dbcache_test.go Loads .env and applies Maven mirror to embedded Postgres.
scripts/common.sh Loads .env and centralizes mirror defaults.
scripts/create-deployment-blueprint.sh Uses configurable kpt function images and env-based Docker Hub mirror.
scripts/create-deployment-config.sh Passes GHCR prefix to deployment blueprint generation.
scripts/create-deployment-kpt.sh Uses configurable kpt function images.
scripts/install-dev-gitea-setup.sh Sources common config and applies mirrors to Gitea setup.
scripts/remove-controller-from-deployment-config.sh Uses configurable kpt function image prefix.
scripts/remove-porch-server-from-deployment-config.sh Uses configurable kpt function image prefix.
scripts/run-load-test.sh Uses configurable kpt function image prefix.
test/Dockerfile Adds usage TODO comment.
test/Makefile Adds usage TODO comment.
Comments suppressed due to low confidence (1)

scripts/create-deployment-kpt.sh:136

  • This is another PORCH_GHCR_PREFIX_URL use in a set -u script that never initializes the variable. A direct run without the env var fails here before deployment customization can proceed.
    --image "${PORCH_GHCR_PREFIX_URL}/starlark:v0.5.5" \

Comment thread Makefile Outdated
Comment thread scripts/remove-porch-server-from-deployment-config.sh
Comment thread scripts/remove-controller-from-deployment-config.sh
Comment thread scripts/run-load-test.sh
Comment thread scripts/create-deployment-blueprint.sh
Comment thread test/Makefile
Comment thread test/Dockerfile
Comment thread build/Dockerfile.etcd Outdated
Comment thread build/Dockerfile.apiserver Outdated
Comment thread scripts/create-deployment-kpt.sh
Copilot AI review requested due to automatic review settings May 18, 2026 16:47
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 7 comments.

Comment thread controllers/Dockerfile
Comment thread build/Dockerfile.etcd Outdated
Comment thread build/Dockerfile.apiserver Outdated
Comment thread test/Dockerfile
Comment thread .env.template Outdated
Comment thread Makefile Outdated
Comment thread scripts/common.sh Outdated
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Copilot AI review requested due to automatic review settings May 19, 2026 09:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (4)

func/Makefile:55

  • Same issue in the push target: the continued lines of this recipe need to be tab-indented as well, otherwise make will not parse this rule correctly.
	docker buildx build --push --tag $(IMAGE_REPO)/$(IMAGE_NAME):$(IMAGE_TAG) \
      --build-arg ALPINE_VERSION=$(ALPINE_VERSION) \
      --build-arg GOLANG_ALPINE_VERSION=$(GOLANG_ALPINE_VERSION) \
	  $$(if [ -n "${DOCKERHUB_MIRROR}" ]; then echo "--build-arg DOCKERHUB_MIRROR=${DOCKERHUB_MIRROR}"; fi) \
	  $$(if [ -n "${PORCH_GHCR_PREFIX_URL}" ]; then echo "--build-arg PORCH_GHCR_PREFIX_URL=${PORCH_GHCR_PREFIX_URL}"; fi) \
      -f ./Dockerfile "$(PORCHDIR)"

controllers/Makefile:33

  • Same parsing issue in push-image docker-push: the continuation lines must start with a tab to be part of the recipe.
	docker buildx build --push --tag $(IMAGE_REPO)/$(IMAGE_NAME):$(IMAGE_TAG) \
      --build-arg ALPINE_VERSION=$(ALPINE_VERSION) \
      --build-arg GOLANG_BOOKWORM_VERSION=$(GOLANG_BOOKWORM_VERSION) \
	  $$(if [ -n "${DOCKERHUB_MIRROR}" ]; then echo "--build-arg DOCKERHUB_MIRROR=${DOCKERHUB_MIRROR}"; fi) \
      -f Dockerfile ..

build/Makefile:35

  • Same parsing issue in push-image docker-push: continuation lines must begin with a tab, otherwise this target won't run.
	docker buildx build --push --tag $(IMAGE_REPO)/$(IMAGE_NAME):$(IMAGE_TAG) \
      --build-arg ALPINE_VERSION=$(ALPINE_VERSION) \
      --build-arg GOLANG_BOOKWORM_VERSION=$(GOLANG_BOOKWORM_VERSION) \
	  $$(if [ -n "${DOCKERHUB_MIRROR}" ]; then echo "--build-arg DOCKERHUB_MIRROR=${DOCKERHUB_MIRROR}"; fi) \
      -f ./Dockerfile "$(PORCHDIR)"

test/Makefile:33

  • Same parsing issue in push-image docker-push: the continued docker buildx build lines must be tab-indented to be part of the recipe.
	docker buildx build --push --tag $(IMAGE_REPO)/$(IMAGE_NAME):$(IMAGE_TAG) \
      --build-arg GOLANG_BOOKWORM_VERSION=$(GOLANG_BOOKWORM_VERSION) \
	  $$(if [ -n "${DOCKERHUB_MIRROR}" ]; then echo "--build-arg DOCKERHUB_MIRROR=${DOCKERHUB_MIRROR}"; fi) \
      -f Dockerfile ..

Comment thread func/Makefile
Comment thread controllers/Makefile
Comment thread build/Makefile
Comment thread test/Makefile
Comment thread scripts/common.sh
Comment thread pkg/cache/dbcache/dbcache_test.go
Comment thread build/Dockerfile.etcd
Comment on lines 15 to 20
ARG ALPINE_VERSION=latest
ARG GOLANG_BOOKWORM_VERSION=latest
ARG DOCKERHUB_MIRROR=docker.io

FROM golang:1.25.7-bookworm AS builder
FROM ${DOCKERHUB_MIRROR}/golang:${GOLANG_BOOKWORM_VERSION} AS builder

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aravindtga was this kept intentionally pinned, or was this just an oversight before?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mozesl-nokia This is intentional. These values are not used by the make targets that build/push images. the actual values are defined in the root Makefile(passed by the respective Makefile/targets). This value will be used only when the Docker build is called without passing GOLANG_BOOKWORM_VERSION build argument.

Comment thread .env.template
@mozesl-nokia
Copy link
Copy Markdown
Contributor Author

Want to point this out from Copilot's comments: Do we want to rename the PORCH_GHCR_PREFIX_URL env var?

Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Copilot AI review requested due to automatic review settings May 20, 2026 08:32
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 2 comments.

Comment thread scripts/common.sh
Comment thread build/Dockerfile.etcd
Comment thread Makefile
export PORCH_GHCR_PREFIX_URL := $(patsubst %/,%,$(PORCH_GHCR_PREFIX_URL))
export DOCKERHUB_MIRROR ?=
# remove '/' suffix
export DOCKERHUB_MIRROR := $(patsubst %/,%,$(DOCKERHUB_MIRROR))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the build-images and push-images targets in make/build.mk be updated to propagate these values?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants