Skip to content

chore(build): build wheels for multiple platforms #281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jericht merged 1 commit into from
Jun 23, 2026
Merged

chore(build): build wheels for multiple platforms #281

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 17 additions & 38 deletions .github/workflows/release_publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,57 +35,36 @@ jobs:
tag: ${{ needs.TagRelease.outputs.tag }}

PreRelease:

@leongdl leongdl Jun 22, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, this is how I reviewed it. I asked my agent to look at the before and after, and sanity check the paths:

Here's the summary of the release workflow changes:
  
  Before (current mainline)
  
  The workflow pipeline is:
  
  1. TagRelease → creates a git tag
  2. UnitTests → runs code quality checks
  3. PreRelease → uses reusable_prerelease.yml (generic pre-release)
  4. Release → uses reusable_release.yml (creates the GitHub Release)
  5. Publish → uses reusable_publish_python.yml (a separate reusable workflow step)
  6. PublishToPyPI → checks out the tag, sets up Python 3.9, installs hatch, runs hatch -v build to build the package locally on a single Ubuntu runner, then publishes the resulting wheel/sdist to PyPI via trusted
  publishing
  
  The key point: the build happens at PyPI-publish time on a single Ubuntu runner using hatch build. This produces a single pure-Python py3-none-any wheel since there was no native code.
  
  After (this PR)
  
  The workflow pipeline becomes:
  
  1. TagRelease → same
  2. UnitTests → same
  3. PreRelease → now uses reusable_maturin_prerelease.yml — a new reusable workflow that builds platform-specific wheels via maturin across multiple OS/arch combinations (macOS x86_64/arm64, Linux x86_64/aarch64,
  Windows amd64/arm64). It also no longer requests id-token/contents: write permissions itself.
  4. Release → same reusable workflow (creates GitHub Release), now with explicit permissions
  5. Publish job is removed entirely
  6. PublishToPyPI → no longer checks out code or builds anything. Instead it downloads the pre-built build-artifact (the multi-platform wheels produced by PreRelease) and publishes them directly to PyPI.
  Dependencies changed from [TagRelease, Publish] to [TagRelease, Release, PreRelease].
  
  Other changes
  
  - _build_backend.py: On Windows, sets CARGO_TARGET_DIR=C:\cargo-target to avoid MSVC linker failures from long paths when building from an extracted sdist.
  - scripts/maturin_build.py: Adds a --version-only flag that patches version into pyproject.toml and _version.py without running maturin — used by CI when maturin-action handles the actual compilation but needs
  the version injected beforehand.

needs: [TagRelease, UnitTests]
uses: OpenJobDescription/.github/.github/workflows/reusable_prerelease.yml@mainline
permissions:
id-token: write
contents: write
secrets: inherit
with:
tag: ${{ needs.TagRelease.outputs.tag }}
needs: [TagRelease, UnitTests]
uses: OpenJobDescription/.github/.github/workflows/reusable_maturin_prerelease.yml@mainline
secrets: inherit
with:
tag: ${{ needs.TagRelease.outputs.tag }}

Release:
needs: [TagRelease, PreRelease]
uses: OpenJobDescription/.github/.github/workflows/reusable_release.yml@mainline
secrets: inherit
permissions:
id-token: write
contents: write
with:
tag: ${{ needs.TagRelease.outputs.tag }}

Publish:
needs: [TagRelease, Release]
uses: OpenJobDescription/.github/.github/workflows/reusable_publish_python.yml@mainline
Comment on lines -57 to -59

@jericht jericht Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step is being removed because we are moving away from the CodeBuild based internal build system.

needs: [TagRelease, PreRelease]
uses: OpenJobDescription/.github/.github/workflows/reusable_release.yml@mainline

@leongdl leongdl Jun 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the key line to call to the shared workflow right?

@jericht jericht Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually an existing release flow, the above one (reusable_maturin_prerelease) is the new reusable workflow added in OpenJobDescription/.github#17

secrets: inherit
permissions:
id-token: write
secrets: inherit
contents: write
with:
tag: ${{ needs.TagRelease.outputs.tag }}
tag: ${{ needs.TagRelease.outputs.tag }}

# PyPI does not support reusable workflows yet
# # See https://github.com/pypi/warehouse/issues/11096
# See https://github.com/pypi/warehouse/issues/11096
PublishToPyPI:
needs: [TagRelease, Publish]
needs: [TagRelease, Release, PreRelease]
Comment thread
godobyte marked this conversation as resolved.
runs-on: ubuntu-latest
environment: release
permissions:
id-token: write
steps:
- name: Checkout
uses: actions/checkout@v6
- uses: actions/download-artifact@v4
with:
ref: ${{ needs.TagRelease.outputs.tag }}
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.9'
- name: Install dependencies
run: |
pip install --upgrade hatch "virtualenv<21"
- name: Build
run: hatch -v build
# # See https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-pypi
name: build-artifact
path: dist
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist
7 changes: 7 additions & 0 deletions _build_backend.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,17 @@
from __future__ import annotations

import contextlib
import os
import shutil
import sys
from pathlib import Path

# Windows MSVC linker fails with LNK1104 when paths exceed MAX_PATH (260 chars).
# Building from an extracted sdist produces deeply nested target directories.
# Redirect Cargo's output to a short path to avoid this.
if sys.platform == "win32" and "CARGO_TARGET_DIR" not in os.environ:
os.environ["CARGO_TARGET_DIR"] = "C:\\cargo-target"

import maturin

REPO_ROOT = Path(__file__).resolve().parent
Expand Down
11 changes: 11 additions & 0 deletions scripts/maturin_build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,17 @@ def main(argv: list[str]) -> int:
)
shutil.move(str(PYPROJECT_BAK), str(PYPROJECT))

# --version-only: inject the VCS version into pyproject.toml and
# _version.py without running maturin. Used in CI when maturin-action
# handles the actual build but needs the version patched beforehand.
if "--version-only" in argv:
version = compute_version()
write_version_file(version)
print(f"Wrote src/openjd/model/_version.py (version {version})")
_patch_pyproject(version)
print(f"Patched pyproject.toml: project.version = {version!r}")
return 0

maturin_args = argv or ["develop"]

version = compute_version()
Expand Down
Loading