Skip to content

fix(ci): restore multi-arch container builds in release workflow#538

Merged
christiangda merged 1 commit into
mainfrom
fix/container-build-qemu-setup
May 24, 2026
Merged

fix(ci): restore multi-arch container builds in release workflow#538
christiangda merged 1 commit into
mainfrom
fix/container-build-qemu-setup

Conversation

@christiangda
Copy link
Copy Markdown
Contributor

Summary

Fixes the Publish Container Images job in the release workflow, which has been failing since v0.44.1 and surfaced again on the v0.45.0 release as "Could not resolve digest for ghcr.io/slashdevops/idp-scim-sync:v0.45.0".

Root cause

When the workflow was migrated from Docker to Podman in ab22744 ("fix: replace Docker with Podman and simplify container image publishing"), the docker/setup-qemu-action step was removed on the assumption that pre-built binaries no longer needed cross-compilation.

They don't — but every RUN line in Containerfile (notably apk add --no-cache --update ca-certificates) still executes under the target architecture. On the amd64 runner, building the arm64 variant therefore needs QEMU user-mode emulation registered with binfmt_misc. Without it:

  1. podman build --platform linux/arm64 died with exec /bin/sh: Exec format error.
  2. No …-linux-arm64 image was created in local storage.
  3. make container-publish then failed: podman manifest add could not find the arm64 image.
  4. No manifest was ever pushed to GHCR.
  5. cosign sign finally turned the job red with "Could not resolve digest for ghcr.io/slashdevops/idp-scim-sync:v0.45.0".

The two upstream make failures were not surfaced as red steps because make ... | tee $GITHUB_STEP_SUMMARY runs under bash without pipefail, so tee's exit 0 masked make's exit 1. That's why the job summary showed ✓ Make container-build, ✓ Make container-publish, and only the final ✗ Cosign sign … step turned red.

Changes

In .github/workflows/container-image.yml:

  • Install qemu-user-static and binfmt-support alongside podman so the kernel can execute arm64 binaries during the build.
  • Add defaults.run.shell: bash -eo pipefail {0} at the job level so future cmd | tee ... failures fail the step instead of being silently swallowed.

Plus a Whats-New.md entry under Unreleased.

No code or release-artifact changes; the multi-arch image set and signing flow are unchanged.

Test plan

  • PR merges cleanly.
  • On the next tag push, the Publish Container Images job:
    • Builds both …-linux-amd64 and …-linux-arm64 images.
    • Pushes a multi-arch manifest to ghcr.io/slashdevops/idp-scim-sync:<tag> and :latest.
    • Successfully resolves the manifest digest and signs it with cosign.
  • (Manual) Re-trigger the Container Image workflow via workflow_dispatch after merge to validate without a tag.
  • (Recovery, separate action) Re-run / re-publish the v0.45.0 container manifest so the existing release is signed.

🤖 Generated with Claude Code

The Publish Container Images job has been failing since v0.44.1 (visible
again on the v0.45.0 release as "Could not resolve digest for
ghcr.io/slashdevops/idp-scim-sync:v0.45.0") because the arm64 image is
never produced.

Root cause: when the workflow was migrated from Docker to Podman in
ab22744, docker/setup-qemu-action was removed on the assumption that
pre-built binaries no longer needed cross-compilation. They don't, but
every RUN line in Containerfile (apk add ca-certificates, etc.) still
executes under the target architecture, which on an amd64 runner
requires QEMU user-mode emulation registered with binfmt_misc. Without
it, `podman build --platform linux/arm64` exits with
"exec /bin/sh: Exec format error", no arm64 image is created,
container-publish then fails to add it to the manifest, and cosign
finally fails to resolve the (never-pushed) digest.

The two upstream make failures were not surfaced as red steps because
`make ... | tee $GITHUB_STEP_SUMMARY` runs under bash without pipefail,
so tee's exit 0 masked the make exit 1.

Changes:
* Install qemu-user-static and binfmt-support alongside podman so the
  kernel can run arm64 binaries during the build.
* Add `defaults.run.shell: bash -eo pipefail {0}` at the job level so
  future `cmd | tee ...` failures fail the step instead of being
  silently swallowed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@christiangda christiangda added this pull request to the merge queue May 24, 2026
Merged via the queue into main with commit 2cf16ba May 24, 2026
6 checks passed
@christiangda christiangda deleted the fix/container-build-qemu-setup branch May 24, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant