Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 156 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# SPDX-FileCopyrightText: 2026 Samudra Authors
#
# SPDX-License-Identifier: Apache-2.0

# Builds the pure-Python samudra wheel + sdist and publishes to PyPI via OIDC
# trusted publishing -- no API token is stored anywhere. Publishing runs only
# after the CPU test suite passes (see the `test` job). Triggers:
# schedule -> weekly nightly dev release (<base>.dev<stamp>)
# push tag v* -> stable release, version from the v<version> tag
# workflow_dispatch -> hand-triggered cut (mode: nightly / stable / smoke)
# pull_request -> build-only smoke when the build script or this
# workflow file changes
# The `smoke` mode is build-only (never publishes); note that every mode can be
# hand-triggered via workflow_dispatch -- "smoke" names what it does, not how it
# starts. One-time trusted-publisher setup is documented in docs/releasing.md.
name: Release

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This release process is inspire'd by the Marin repo.


on:
workflow_dispatch:
inputs:
mode:
description: "Build mode"
type: choice
options: [nightly, stable, smoke]
default: smoke
version:
description: "Stable-cut version, e.g. 0.0.1 (required for stable; ignored for nightly/smoke)"
type: string
schedule:
- cron: "0 6 * * 1" # 06:00 UTC every Monday
push:
tags:
- "v*"
pull_request:
paths:
- "scripts/package.py"
- ".github/workflows/release.yml"

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: false # don't kill an in-flight publish

permissions:
contents: read

jobs:
resolve:
runs-on: ubuntu-latest
outputs:
mode: ${{ steps.pick.outputs.mode }}
version: ${{ steps.resolved.outputs.version }}
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
with:
fetch-depth: 0 # setuptools-scm + `git describe` need full tag history

# Hoist user-supplied dispatch inputs into env so they reach the shell as
# variables, not as `${{ ... }}` template substitutions -- free-form
# inputs interpolated into a shell script are an injection vector.
- id: pick
env:
INPUT_MODE: ${{ github.event.inputs.mode }}
INPUT_VERSION: ${{ github.event.inputs.version }}
run: |
set -euo pipefail
if [[ "${GITHUB_EVENT_NAME}" == "push" && "${GITHUB_REF}" == refs/tags/v* ]]; then
echo "mode=stable" >> "$GITHUB_OUTPUT"
echo "input_version=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT"
elif [[ "${GITHUB_EVENT_NAME}" == "schedule" ]]; then
echo "mode=nightly" >> "$GITHUB_OUTPUT"
echo "input_version=" >> "$GITHUB_OUTPUT"
elif [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was very confused how manual builds (ie workflow_dispatch) may or may not have a mode of manual. Should we maybe call that mode "smoke" or something rather than manual? Since you can manually trigger a stable or nightly build, too.

echo "mode=${INPUT_MODE}" >> "$GITHUB_OUTPUT"
# The version field is only meaningful for a stable cut. For nightly
# and manual dispatches, discard it so the version is always computed
# -- otherwise a value left in the field would be stamped verbatim
# and published as a bogus "nightly", burning a real PyPI version.
if [[ "${INPUT_MODE}" == "stable" ]]; then
echo "input_version=${INPUT_VERSION}" >> "$GITHUB_OUTPUT"
else
echo "input_version=" >> "$GITHUB_OUTPUT"
fi
else
# pull_request: build-only smoke
echo "mode=smoke" >> "$GITHUB_OUTPUT"
echo "input_version=" >> "$GITHUB_OUTPUT"
fi

# Resolve the version exactly once for the whole run so the build job
# stamps the identical value -- a run straddling midnight UTC would
# otherwise compute a different nightly date in build vs resolve.
- id: resolved
env:
MODE: ${{ steps.pick.outputs.mode }}
INPUT_VERSION: ${{ steps.pick.outputs.input_version }}
run: |
python3 scripts/package.py --mode "$MODE" --version "$INPUT_VERSION" --resolve-only

build:
needs: resolve
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
with:
fetch-depth: 0 # setuptools-scm + `git describe` need full tag history
- uses: astral-sh/setup-uv@d0d8abe699bfb85fec6de9f7adb5ae17292296ff

- name: Build wheel + sdist
env:
MODE: ${{ needs.resolve.outputs.mode }}
VERSION: ${{ needs.resolve.outputs.version }}
run: |
python3 scripts/package.py --mode "$MODE" --version "$VERSION"

# Fail fast on broken metadata before we ever reach the publish step.
- name: Check distribution metadata
run: uvx twine check dist/*

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: samudra-dist
path: dist/*
retention-days: 14

# Gate every publish on the CPU test suite. A tag push doesn't otherwise run
# test.yml (it's push-to-main / PR only), so call it here as a required check.
test:
needs: resolve
if: |
github.event_name != 'pull_request'
&& needs.resolve.outputs.mode != 'smoke'
uses: ./.github/workflows/test.yml

publish:
# Smoke mode produces local-version identifiers (`X.Y.Z+smoke.<sha>`) which
# PyPI rejects, so smoke cuts -- and every pull_request build -- are
# build-only regardless of how they were triggered. Publishing also waits on
# the test job, so a red test suite blocks the upload.
if: |
github.event_name != 'pull_request'
&& needs.resolve.outputs.mode != 'smoke'
needs: [resolve, build, test]
runs-on: ubuntu-latest
environment: pypi-publish
permissions:
id-token: write # mint the OIDC token for trusted publishing
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: samudra-dist
path: dist

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@ba38be9e461d3875417946c167d0b5f3d385a247
with:
packages-dir: dist/
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ on:
branches:
- "**"
merge_group:
workflow_call: # let the Release workflow gate publishes on this suite

concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,6 @@ build/
.claude/
site/
profiles/

# Build artifacts
dist/
23 changes: 21 additions & 2 deletions docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,28 @@ SPDX-License-Identifier: CC-BY-4.0
- Python 3.12
- [uv](https://docs.astral.sh/uv/) package manager

## Setup
## Install from PyPI

Clone the repository and install dependencies:
Samudra is pure Python, so one wheel covers every platform. The GPU custom
kernels are opt-in:

```bash
# Install with `uv` (recommended)
uv add samudra # CPU (default)
uv add "samudra[cuda]" # adds flash-attn, flash-perceiver, torchvision
uv add samudra --prerelease=allow # latest nightly dev build
# Install with `pip`
pip install samudra # CPU (default)
pip install "samudra[cuda]" # adds flash-attn, flash-perceiver, torchvision
pip install --pre samudra # latest nightly dev build
```

The `cuda` extra compiles native kernels against your local CUDA + `torch`; see
[Releasing to PyPI](../releasing.md#installing-the-package) for the details.

## Development setup

To work on Samudra itself, clone the repository and install dependencies:

```bash
git clone https://github.com/m2lines/Samudra.git
Expand Down
126 changes: 126 additions & 0 deletions docs/releasing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
<!--
SPDX-FileCopyrightText: 2026 Samudra Authors

SPDX-License-Identifier: CC-BY-4.0
-->

# Releasing to PyPI

Samudra is published to [PyPI](https://pypi.org/project/samudra/) as a single
pure-Python wheel by the [`Release`](https://github.com/m2lines/Samudra/actions/workflows/release.yml)
workflow. It authenticates with [OIDC trusted publishing](https://docs.pypi.org/trusted-publishers/),
so no API token is stored anywhere.

## Installing the package

Samudra itself is pure Python, so one universal wheel serves every platform.
The GPU custom kernels are opt-in.

```bash
# CPU (default) — everything except the compiled GPU kernels
uv add samudra
pip install samudra

# GPU — adds flash-attn, flash-perceiver, and torchvision, which compile
# against your local CUDA + torch at install time
uv add "samudra[cuda]"
pip install "samudra[cuda]"

# Latest nightly dev build
uv add samudra --prerelease=allow
pip install --pre samudra
```

The `cuda` extra builds native kernels, so it needs a CUDA toolchain and a
matching `torch` already present. With `uv` the `[tool.uv]` build settings in
`pyproject.toml` handle this automatically; with plain `pip` you typically want
`pip install --no-build-isolation "samudra[cuda]"` in an environment that
already has `torch`.

Installing exposes a `samudra` console command that mirrors the module entry
points, so you don't need a checkout to run a task against your own config:

```bash
samudra train path/to/train.yaml --experiment.data_root $DATA_PATH
samudra eval path/to/eval.yaml --ckpt_path path/to/checkpoint
samudra viz path/to/viz.yaml
```

The example configs under `configs/` are not yet shipped in the wheel — pass a
path to your own YAML (or one from a checkout). Packaging the presets so
`samudra train samudra_om4/train.yaml` resolves them is planned as a follow-up.

## How versions are cut

The version is owned by [setuptools-scm](https://setuptools-scm.readthedocs.io/):
there is **no** `version = "..."` field to maintain — a git tag *is* the version.
`[tool.setuptools_scm]` in `pyproject.toml` configures it, and `samudra.__version__`
is available at runtime. The release paths differ only in what version reaches
the build:

| Trigger | Mode | Version | Published? |
| --- | --- | --- | --- |
| Push a `v*` tag | `stable` | the tag, e.g. `v1.0.0` → `1.0.0` (setuptools-scm) | ✅ PyPI |
| Weekly `schedule` (Mon 06:00 UTC) | `nightly` | `<next-patch>.dev<YYYYMMDDhhmm>` | ✅ PyPI |
| `workflow_dispatch` → `nightly`/`stable` | as chosen | as above | ✅ PyPI |
| `workflow_dispatch` → `smoke` | `smoke` | `<next-patch>+smoke.<sha>` | ❌ build-only |
| Pull request touching the script/workflow | `smoke` | — | ❌ build-only |
| Local editable install (`uv sync`) | — | `<next-patch>.dev<N>` from git | n/a |

The scheduled dev release is weekly rather than daily — Samudra doesn't turn over
enough in a day to warrant one, and the timestamp still makes each build unique.
The `smoke` mode is build-only: it names what the build *does* (a no-publish
check), not how it's triggered — every mode, `smoke` included, can be started by
hand from **Actions → Release → Run workflow**.

On a tagged commit setuptools-scm derives the version straight from the tag. For
the two synthetic modes, `scripts/package.py` computes the version and hands it
to setuptools-scm via `SETUPTOOLS_SCM_PRETEND_VERSION_FOR_SAMUDRA` — it never
edits a tracked file. A nightly's base is one patch above the most recent `v*`
tag (or `fallback_version` before the first tag), and its UTC timestamp keeps
every nightly unique and PEP 440-ordered *above* the last release, so `--pre`
resolves them.

Every publish (stable **and** nightly) waits on the CPU test suite: the release
workflow calls `test.yml` as a required `test` job, so a red suite blocks the
upload. A tag push doesn't otherwise run the tests, which is why the release
workflow invokes them itself.

### Cutting a stable release

```bash
# Just tag and push — no version bump anywhere:
git tag v1.1.0
git push origin v1.1.0
```

The tag push runs `resolve → test → build → publish`, uploading `samudra 1.1.0`
to PyPI. To dry-run first, use **Actions → Release → Run workflow → mode: smoke**;
that builds and runs `twine check` without publishing.

!!! note "Before the first tag"
Comment thread
alxmrs marked this conversation as resolved.
The repository has no `v*` tags yet, so the "last release" falls back to
`0.0.0` (`fallback_version` in `[tool.setuptools_scm]`, mirrored by
`FALLBACK_VERSION` in `scripts/package.py` — keep the two in sync). Builds
therefore target `0.0.1` (e.g. a nightly is `0.0.1.dev<stamp>`). Cutting the
first tag, **`v0.0.1`**, makes the tag the single source of truth from then
on.

## One-time trusted-publisher setup
Comment thread
alxmrs marked this conversation as resolved.

Before the first publish, register the repository as a trusted publisher on
PyPI (a maintainer with project-owner rights does this once):

1. Create the project on PyPI, or use [pending publishers](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/)
to reserve the name `samudra` before the first upload.
2. On the project's **Settings → Publishing** page, add a GitHub Actions
publisher with:
- **Owner**: `m2lines`
- **Repository**: `Samudra`
- **Workflow name**: `release.yml`
- **Environment**: `pypi-publish`
3. In the GitHub repo, create an environment named `pypi-publish`
(**Settings → Environments**). Optionally add required reviewers so stable
releases need an approval before the publish job runs.

No secrets are needed — the publish job mints a short-lived OIDC token per run.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ nav:
- Torch: torch.md
- Development:
- Contributing: contributing.md
- Releasing: releasing.md
- API Reference:
- Models:
- Base Model: models/base.md
Expand Down
30 changes: 28 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,10 @@

[build-system]
build-backend = "setuptools.build_meta"
requires = [ "setuptools", "wheel" ]
requires = [ "setuptools", "setuptools-scm>=8", "wheel" ]

[project]
name = "samudra"
version = "1.0"
description = "Train and evaluate ocean model emulators"
readme = "README.md"
keywords = [ "climate", "emulation", "machine learning", "ocean" ]
Expand All @@ -23,6 +22,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dynamic = [ "version" ] # derived from git tags by setuptools-scm
Comment thread
alxmrs marked this conversation as resolved.
dependencies = [
"aiohttp>=3.11.13",
"cftime>=1.6.4.post1",
Expand Down Expand Up @@ -57,6 +57,13 @@ optional-dependencies.cuda = [
"torchvision>=0.24.0a0",
]

urls.Documentation = "https://m2lines.github.io/Samudra/docs/"
urls.Homepage = "https://m2lines.github.io/Samudra/"
urls.Issues = "https://github.com/m2lines/Samudra/issues"
urls.Repository = "https://github.com/m2lines/Samudra"
# `samudra train|eval|viz CONFIG` for installed users; mirrors `python -m samudra.train`.
scripts.samudra = "samudra.cli:main"

[dependency-groups]
dev = [
"beartype>=0.20",
Expand Down Expand Up @@ -91,6 +98,25 @@ docs = [
"zensical",
]

[tool.setuptools]
package-dir = { "" = "src" }
package-data = { samudra = [ "py.typed" ] }

# Pin discovery to src/ so setuptools auto-discovery can't sweep stray
# top-level dirs (e.g. a stale build/lib/) into the wheel.
[tool.setuptools.packages.find]
where = [ "src" ]

# The version comes from git tags (v1.2.3 -> 1.2.3); commits past a tag get a
# `.devN` suffix. no-local-version drops the `+g<sha>` segment PyPI rejects.
# fallback_version is the last released version before the first tag exists
# (0.0.0 = none yet, so builds target 0.0.1); guess-next-dev bumps it. Keep it
# in sync with FALLBACK_VERSION in scripts/package.py.

[tool.setuptools_scm]
local_scheme = "no-local-version"
fallback_version = "0.0.0"

[tool.ruff]
lint.select = [ "B006", "D", "E", "F", "I", "UP", "W" ]
# F722: https://docs.kidger.site/jaxtyping/faq/#flake8-or-ruff-are-throwing-an-error
Expand Down
Loading
Loading