-
Notifications
You must be signed in to change notification settings - Fork 1
Add PyPI release workflow with weekly and stable channels #817
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
24b8d05
Add PyPI release workflow with nightly and stable channels
alxmrs 7e8e5d0
Make room for initial version of 0.0.1
alxmrs d7eff9e
Prevent a dispatched nightly from publishing an arbitrary version
alxmrs e321dd3
Merge branch 'main' into u/alxmrs/pypi
alxmrs b66e25b
Point repo URLs and trusted publisher at m2lines/Samudra
alxmrs 8537600
Address Jesse's review: test-gated publish, smoke mode, weekly, CLI
alxmrs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,156 @@ | ||
| # SPDX-FileCopyrightText: 2026 Samudra Authors | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # Builds the pure-Python samudra wheel + sdist and publishes to PyPI via OIDC | ||
| # trusted publishing -- no API token is stored anywhere. Publishing runs only | ||
| # after the CPU test suite passes (see the `test` job). Triggers: | ||
| # schedule -> weekly nightly dev release (<base>.dev<stamp>) | ||
| # push tag v* -> stable release, version from the v<version> tag | ||
| # workflow_dispatch -> hand-triggered cut (mode: nightly / stable / smoke) | ||
| # pull_request -> build-only smoke when the build script or this | ||
| # workflow file changes | ||
| # The `smoke` mode is build-only (never publishes); note that every mode can be | ||
| # hand-triggered via workflow_dispatch -- "smoke" names what it does, not how it | ||
| # starts. One-time trusted-publisher setup is documented in docs/releasing.md. | ||
| name: Release | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| mode: | ||
| description: "Build mode" | ||
| type: choice | ||
| options: [nightly, stable, smoke] | ||
| default: smoke | ||
| version: | ||
| description: "Stable-cut version, e.g. 0.0.1 (required for stable; ignored for nightly/smoke)" | ||
| type: string | ||
| schedule: | ||
| - cron: "0 6 * * 1" # 06:00 UTC every Monday | ||
| push: | ||
| tags: | ||
| - "v*" | ||
| pull_request: | ||
| paths: | ||
| - "scripts/package.py" | ||
| - ".github/workflows/release.yml" | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} | ||
| cancel-in-progress: false # don't kill an in-flight publish | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| resolve: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| mode: ${{ steps.pick.outputs.mode }} | ||
| version: ${{ steps.resolved.outputs.version }} | ||
| steps: | ||
| - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 | ||
| with: | ||
| fetch-depth: 0 # setuptools-scm + `git describe` need full tag history | ||
|
|
||
| # Hoist user-supplied dispatch inputs into env so they reach the shell as | ||
| # variables, not as `${{ ... }}` template substitutions -- free-form | ||
| # inputs interpolated into a shell script are an injection vector. | ||
| - id: pick | ||
| env: | ||
| INPUT_MODE: ${{ github.event.inputs.mode }} | ||
| INPUT_VERSION: ${{ github.event.inputs.version }} | ||
| run: | | ||
| set -euo pipefail | ||
| if [[ "${GITHUB_EVENT_NAME}" == "push" && "${GITHUB_REF}" == refs/tags/v* ]]; then | ||
| echo "mode=stable" >> "$GITHUB_OUTPUT" | ||
| echo "input_version=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT" | ||
| elif [[ "${GITHUB_EVENT_NAME}" == "schedule" ]]; then | ||
| echo "mode=nightly" >> "$GITHUB_OUTPUT" | ||
| echo "input_version=" >> "$GITHUB_OUTPUT" | ||
| elif [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was very confused how manual builds (ie workflow_dispatch) may or may not have a mode of manual. Should we maybe call that mode "smoke" or something rather than manual? Since you can manually trigger a stable or nightly build, too. |
||
| echo "mode=${INPUT_MODE}" >> "$GITHUB_OUTPUT" | ||
| # The version field is only meaningful for a stable cut. For nightly | ||
| # and manual dispatches, discard it so the version is always computed | ||
| # -- otherwise a value left in the field would be stamped verbatim | ||
| # and published as a bogus "nightly", burning a real PyPI version. | ||
| if [[ "${INPUT_MODE}" == "stable" ]]; then | ||
| echo "input_version=${INPUT_VERSION}" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "input_version=" >> "$GITHUB_OUTPUT" | ||
| fi | ||
| else | ||
| # pull_request: build-only smoke | ||
| echo "mode=smoke" >> "$GITHUB_OUTPUT" | ||
| echo "input_version=" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| # Resolve the version exactly once for the whole run so the build job | ||
| # stamps the identical value -- a run straddling midnight UTC would | ||
| # otherwise compute a different nightly date in build vs resolve. | ||
| - id: resolved | ||
| env: | ||
| MODE: ${{ steps.pick.outputs.mode }} | ||
| INPUT_VERSION: ${{ steps.pick.outputs.input_version }} | ||
| run: | | ||
| python3 scripts/package.py --mode "$MODE" --version "$INPUT_VERSION" --resolve-only | ||
|
|
||
| build: | ||
| needs: resolve | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 | ||
| with: | ||
| fetch-depth: 0 # setuptools-scm + `git describe` need full tag history | ||
| - uses: astral-sh/setup-uv@d0d8abe699bfb85fec6de9f7adb5ae17292296ff | ||
|
|
||
| - name: Build wheel + sdist | ||
| env: | ||
| MODE: ${{ needs.resolve.outputs.mode }} | ||
| VERSION: ${{ needs.resolve.outputs.version }} | ||
| run: | | ||
| python3 scripts/package.py --mode "$MODE" --version "$VERSION" | ||
|
|
||
| # Fail fast on broken metadata before we ever reach the publish step. | ||
| - name: Check distribution metadata | ||
| run: uvx twine check dist/* | ||
|
|
||
| - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a | ||
| with: | ||
| name: samudra-dist | ||
| path: dist/* | ||
| retention-days: 14 | ||
|
|
||
| # Gate every publish on the CPU test suite. A tag push doesn't otherwise run | ||
| # test.yml (it's push-to-main / PR only), so call it here as a required check. | ||
| test: | ||
| needs: resolve | ||
| if: | | ||
| github.event_name != 'pull_request' | ||
| && needs.resolve.outputs.mode != 'smoke' | ||
| uses: ./.github/workflows/test.yml | ||
|
|
||
| publish: | ||
| # Smoke mode produces local-version identifiers (`X.Y.Z+smoke.<sha>`) which | ||
| # PyPI rejects, so smoke cuts -- and every pull_request build -- are | ||
| # build-only regardless of how they were triggered. Publishing also waits on | ||
| # the test job, so a red test suite blocks the upload. | ||
| if: | | ||
| github.event_name != 'pull_request' | ||
| && needs.resolve.outputs.mode != 'smoke' | ||
| needs: [resolve, build, test] | ||
| runs-on: ubuntu-latest | ||
| environment: pypi-publish | ||
| permissions: | ||
| id-token: write # mint the OIDC token for trusted publishing | ||
| steps: | ||
| - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c | ||
| with: | ||
| name: samudra-dist | ||
| path: dist | ||
|
|
||
| - name: Publish to PyPI | ||
| uses: pypa/gh-action-pypi-publish@ba38be9e461d3875417946c167d0b5f3d385a247 | ||
| with: | ||
| packages-dir: dist/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -48,3 +48,6 @@ build/ | |
| .claude/ | ||
| site/ | ||
| profiles/ | ||
|
|
||
| # Build artifacts | ||
| dist/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| <!-- | ||
| SPDX-FileCopyrightText: 2026 Samudra Authors | ||
|
|
||
| SPDX-License-Identifier: CC-BY-4.0 | ||
| --> | ||
|
|
||
| # Releasing to PyPI | ||
|
|
||
| Samudra is published to [PyPI](https://pypi.org/project/samudra/) as a single | ||
| pure-Python wheel by the [`Release`](https://github.com/m2lines/Samudra/actions/workflows/release.yml) | ||
| workflow. It authenticates with [OIDC trusted publishing](https://docs.pypi.org/trusted-publishers/), | ||
| so no API token is stored anywhere. | ||
|
|
||
| ## Installing the package | ||
|
|
||
| Samudra itself is pure Python, so one universal wheel serves every platform. | ||
| The GPU custom kernels are opt-in. | ||
|
|
||
| ```bash | ||
| # CPU (default) — everything except the compiled GPU kernels | ||
| uv add samudra | ||
| pip install samudra | ||
|
|
||
| # GPU — adds flash-attn, flash-perceiver, and torchvision, which compile | ||
| # against your local CUDA + torch at install time | ||
| uv add "samudra[cuda]" | ||
| pip install "samudra[cuda]" | ||
|
|
||
| # Latest nightly dev build | ||
| uv add samudra --prerelease=allow | ||
| pip install --pre samudra | ||
| ``` | ||
|
|
||
| The `cuda` extra builds native kernels, so it needs a CUDA toolchain and a | ||
| matching `torch` already present. With `uv` the `[tool.uv]` build settings in | ||
| `pyproject.toml` handle this automatically; with plain `pip` you typically want | ||
| `pip install --no-build-isolation "samudra[cuda]"` in an environment that | ||
| already has `torch`. | ||
|
|
||
| Installing exposes a `samudra` console command that mirrors the module entry | ||
| points, so you don't need a checkout to run a task against your own config: | ||
|
|
||
| ```bash | ||
| samudra train path/to/train.yaml --experiment.data_root $DATA_PATH | ||
| samudra eval path/to/eval.yaml --ckpt_path path/to/checkpoint | ||
| samudra viz path/to/viz.yaml | ||
| ``` | ||
|
|
||
| The example configs under `configs/` are not yet shipped in the wheel — pass a | ||
| path to your own YAML (or one from a checkout). Packaging the presets so | ||
| `samudra train samudra_om4/train.yaml` resolves them is planned as a follow-up. | ||
|
|
||
| ## How versions are cut | ||
|
|
||
| The version is owned by [setuptools-scm](https://setuptools-scm.readthedocs.io/): | ||
| there is **no** `version = "..."` field to maintain — a git tag *is* the version. | ||
| `[tool.setuptools_scm]` in `pyproject.toml` configures it, and `samudra.__version__` | ||
| is available at runtime. The release paths differ only in what version reaches | ||
| the build: | ||
|
|
||
| | Trigger | Mode | Version | Published? | | ||
| | --- | --- | --- | --- | | ||
| | Push a `v*` tag | `stable` | the tag, e.g. `v1.0.0` → `1.0.0` (setuptools-scm) | ✅ PyPI | | ||
| | Weekly `schedule` (Mon 06:00 UTC) | `nightly` | `<next-patch>.dev<YYYYMMDDhhmm>` | ✅ PyPI | | ||
| | `workflow_dispatch` → `nightly`/`stable` | as chosen | as above | ✅ PyPI | | ||
| | `workflow_dispatch` → `smoke` | `smoke` | `<next-patch>+smoke.<sha>` | ❌ build-only | | ||
| | Pull request touching the script/workflow | `smoke` | — | ❌ build-only | | ||
| | Local editable install (`uv sync`) | — | `<next-patch>.dev<N>` from git | n/a | | ||
|
|
||
| The scheduled dev release is weekly rather than daily — Samudra doesn't turn over | ||
| enough in a day to warrant one, and the timestamp still makes each build unique. | ||
| The `smoke` mode is build-only: it names what the build *does* (a no-publish | ||
| check), not how it's triggered — every mode, `smoke` included, can be started by | ||
| hand from **Actions → Release → Run workflow**. | ||
|
|
||
| On a tagged commit setuptools-scm derives the version straight from the tag. For | ||
| the two synthetic modes, `scripts/package.py` computes the version and hands it | ||
| to setuptools-scm via `SETUPTOOLS_SCM_PRETEND_VERSION_FOR_SAMUDRA` — it never | ||
| edits a tracked file. A nightly's base is one patch above the most recent `v*` | ||
| tag (or `fallback_version` before the first tag), and its UTC timestamp keeps | ||
| every nightly unique and PEP 440-ordered *above* the last release, so `--pre` | ||
| resolves them. | ||
|
|
||
| Every publish (stable **and** nightly) waits on the CPU test suite: the release | ||
| workflow calls `test.yml` as a required `test` job, so a red suite blocks the | ||
| upload. A tag push doesn't otherwise run the tests, which is why the release | ||
| workflow invokes them itself. | ||
|
|
||
| ### Cutting a stable release | ||
|
|
||
| ```bash | ||
| # Just tag and push — no version bump anywhere: | ||
| git tag v1.1.0 | ||
| git push origin v1.1.0 | ||
| ``` | ||
|
|
||
| The tag push runs `resolve → test → build → publish`, uploading `samudra 1.1.0` | ||
| to PyPI. To dry-run first, use **Actions → Release → Run workflow → mode: smoke**; | ||
| that builds and runs `twine check` without publishing. | ||
|
|
||
| !!! note "Before the first tag" | ||
|
alxmrs marked this conversation as resolved.
|
||
| The repository has no `v*` tags yet, so the "last release" falls back to | ||
| `0.0.0` (`fallback_version` in `[tool.setuptools_scm]`, mirrored by | ||
| `FALLBACK_VERSION` in `scripts/package.py` — keep the two in sync). Builds | ||
| therefore target `0.0.1` (e.g. a nightly is `0.0.1.dev<stamp>`). Cutting the | ||
| first tag, **`v0.0.1`**, makes the tag the single source of truth from then | ||
| on. | ||
|
|
||
| ## One-time trusted-publisher setup | ||
|
alxmrs marked this conversation as resolved.
|
||
|
|
||
| Before the first publish, register the repository as a trusted publisher on | ||
| PyPI (a maintainer with project-owner rights does this once): | ||
|
|
||
| 1. Create the project on PyPI, or use [pending publishers](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/) | ||
| to reserve the name `samudra` before the first upload. | ||
| 2. On the project's **Settings → Publishing** page, add a GitHub Actions | ||
| publisher with: | ||
| - **Owner**: `m2lines` | ||
| - **Repository**: `Samudra` | ||
| - **Workflow name**: `release.yml` | ||
| - **Environment**: `pypi-publish` | ||
| 3. In the GitHub repo, create an environment named `pypi-publish` | ||
| (**Settings → Environments**). Optionally add required reviewers so stable | ||
| releases need an approval before the publish job runs. | ||
|
|
||
| No secrets are needed — the publish job mints a short-lived OIDC token per run. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This release process is inspire'd by the Marin repo.