Add PyPI release workflow with nightly and stable channels by alxmrs · Pull Request #817 · m2lines/Samudra

alxmrs · 2026-07-22T23:36:17Z

Summary

Publishes samudra to PyPI as a single pure-Python wheel via a new Release workflow, modeled on marin's release workflow (resolve → build → publish) and adapted to a single-package repo.

Release channels

Trigger	Mode	Version	Published?
Push a `v*` tag	stable	the tag, e.g. `v0.0.1` → `0.0.1`	✅ PyPI
`schedule` (06:00 UTC daily)	nightly	`<next-patch>.dev<YYYYMMDDhhmm>`	✅ PyPI
`workflow_dispatch`	nightly / stable / manual	computed (nightly/manual) or the version input (stable only)	✅ (not manual)
Pull request touching the script/workflow	manual	—	❌ build-only smoke

A manual nightly dispatch is supported: choose mode: nightly and leave the version field blank to build and publish a real 0.0.1.dev<stamp>.

Versioning (setuptools-scm, hybrid)

Version is delegated to setuptools-scm: a git tag is the version, so there's no version = field to maintain and samudra.__version__ works at runtime. scripts/package.py only computes the two synthetic versions (timestamped nightly, manual smoke) and injects them through scm's own SETUPTOOLS_SCM_PRETEND_VERSION_FOR_SAMUDRA override — it never mutates a tracked file.

The timestamped nightly guarantees a unique, PEP 440-ordered dev release even on days with no new commits, which setuptools-scm's commit-distance schemes cannot provide. (Cutting the timestamp logic into a custom version_scheme would either reintroduce same-commit collisions or break scm's same-commit-→-same-version determinism, so it stays in the script.)

Before the first tag, the "last release" falls back to 0.0.0 (fallback_version in [tool.setuptools_scm], mirrored by FALLBACK_VERSION in the script), so builds target 0.0.1.

Publish safety

A dispatched nightly/manual run ignores the version input field — only a stable cut consumes it. This closes a footgun where a stray value left in the field would otherwise be stamped verbatim and published as a bogus non-dev release, permanently burning a PyPI version. As a backstop, the script also refuses to build a nightly without a .dev suffix or a manual smoke without a +manual segment, failing loudly before anything is emitted or published.

CPU vs GPU

Samudra is pure Python, so one universal py3-none-any wheel serves everyone. CPU is the default (pip install samudra); the GPU custom kernels (flash-attn, flash-perceiver, torchvision) stay in the existing cuda extra and compile on the user's machine — nothing GPU-specific enters the wheel.

Security / hardening (per #766)

OIDC trusted publishing — no PyPI token stored; publish job scoped to the pypi-publish environment.
All actions pinned to full commit SHAs.
Top-level permissions: contents: read; id-token: write scoped only to the publish job.

Also included

Pin package discovery to src/ so setuptools auto-discovery can't sweep a stale build/lib/ into the wheel (this was really happening in local builds); ship py.typed; add [project.urls].
Docs: new docs/releasing.md (process, CPU/GPU install, one-time trusted-publisher setup) and a PyPI install section in docs/getting-started/installation.md.

Verification

resolve for all modes, real builds (nightly/stable/manual), and a plain untagged build — all produce a single samudra wheel, twine check PASSED, no stray ocean_emulators.
Publish-safety: a nightly/manual run with a stray version input is rejected (exit 1, nothing emitted); a blank-version nightly dispatch resolves to 0.0.1.dev<stamp>.
Local editable install derives 0.0.1.dev<N> from git automatically.
pre-commit passes on all changed files (ruff, ruff-format, mypy, check-yaml, pyproject-fmt, schemas). uv lock --check clean.

Follow-up required before first publish (maintainer, one-time)

Register the repo as a PyPI trusted publisher (Owner m2lines, Repo Samudra, Workflow release.yml, Environment pypi-publish) — use the pending-publisher flow to reserve the free samudra name.
Create the pypi-publish GitHub environment (optionally with required reviewers).
Tag v0.0.1 to cut the first stable release. Steps documented in docs/releasing.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V9qeRacC2ZdJ4DCbs6SM8t

Publish `samudra` to PyPI as a single pure-Python wheel via a new Release workflow, modeled on marin's resolve -> build -> publish structure and adapted to a single-package repo. Versioning is delegated to setuptools-scm: a git tag *is* the version, so there is no `version =` field to maintain and `samudra.__version__` works at runtime. `scripts/package.py` only computes the two synthetic versions (timestamped nightly, manual smoke) and injects them via SETUPTOOLS_SCM_PRETEND_VERSION_FOR_ SAMUDRA — it never mutates a tracked file. The timestamped nightly guarantees a unique, PEP 440-ordered dev release even on days with no new commits, which setuptools-scm's commit-distance schemes cannot provide. CPU vs GPU: Samudra is pure Python, so one universal py3-none-any wheel serves everyone. CPU is the default (`pip install samudra`); the GPU custom kernels (flash-attn, flash-perceiver, torchvision) stay in the existing `cuda` extra and compile on the user's machine — nothing GPU-specific enters the wheel. Release channels: - schedule (06:00 UTC daily) -> nightly dev release - push tag v* -> stable release (version from the tag) - workflow_dispatch -> manual nightly/stable/manual cut - pull_request (script/wf) -> build-only smoke Details: - Authenticates to PyPI with OIDC trusted publishing (no stored token); publish job is scoped to the `pypi-publish` environment. - Hardened per PR #766: all actions pinned to full commit SHAs, top-level `contents: read` with `id-token: write` scoped only to the publish job. - Pin package discovery to src/ so setuptools auto-discovery can't sweep a stale build/lib/ into the wheel; ship py.typed; add project URLs. - Docs: new docs/releasing.md (process, CPU/GPU install, trusted-publisher setup) and a PyPI install section in getting-started/installation.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V9qeRacC2ZdJ4DCbs6SM8t

alxmrs · 2026-07-22T23:40:50Z

+#   pull_request      -> build-only smoke when the build script or this
+#                        workflow file changes
+# One-time trusted-publisher setup is documented in docs/releasing.md.
+name: Release


This release process is inspire'd by the Marin repo.

alxmrs · 2026-07-22T23:41:43Z

+   to reserve the name `samudra` before the first upload.
+2. On the project's **Settings → Publishing** page, add a GitHub Actions
+   publisher with:
+     - **Owner**: `Open-Athena`


Should this be m2lines?

alxmrs · 2026-07-22T23:44:17Z

+from importlib.metadata import PackageNotFoundError, version
+
+try:
+    __version__ = version("samudra")
+except PackageNotFoundError:  # not installed (e.g. running from a bare source tree)
+    __version__ = "0.0.0+unknown"
+
+__all__ = ["__version__"]


This allows us to call samudra.__version__.

alxmrs · 2026-07-22T23:44:50Z

  "Programming Language :: Python :: 3.12",
  "Topic :: Scientific/Engineering :: Artificial Intelligence",
 ]
+dynamic = [ "version" ] # derived from git tags by setuptools-scm


Should we make our initial tagged version 0.0.1?

0.0.1 looks good to me!

alxmrs · 2026-07-23T21:09:00Z

@codex may I have your review?

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e8e5d0a43

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-23T21:12:31Z

+            echo "input_version=" >> "$GITHUB_OUTPUT"
+          elif [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then
+            echo "mode=${INPUT_MODE}" >> "$GITHUB_OUTPUT"
+            echo "input_version=${INPUT_VERSION}" >> "$GITHUB_OUTPUT"


Ignore dispatch versions unless stable is selected

When a manual run selects nightly but the version input is non-empty, this forwards that value into scripts/package.py, whose main path uses any explicit --version verbatim for every mode; because the publish job only checks that the mode is not manual, that run publishes an arbitrary non-dev version to PyPI instead of a nightly. This can permanently consume the wrong PyPI version if an operator leaves a stable version in the field while choosing nightly.

Useful? React with 👍 / 👎.

A workflow_dispatch that selected nightly but left a value in the version field forwarded that value into scripts/package.py, whose main path uses any explicit --version verbatim for every mode. Since the publish job only checks that the mode is not manual, such a run would publish a bogus non-dev version (e.g. 2.5.0) to PyPI, permanently burning that release number. Fix at both layers: - Workflow: the resolve `pick` step only forwards the dispatch version input for a stable cut; for nightly/manual it clears it so the version is always computed. This also enables the intended manual-nightly dispatch (select nightly, leave version blank) to publish a real .dev build. - Script: refuse to build a nightly without a `.dev` suffix or a manual smoke without a `+manual` segment, so a mislabeled version can never be stamped and published even if the workflow regresses. The check runs before emitting to $GITHUB_OUTPUT, so the resolve job fails loudly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V9qeRacC2ZdJ4DCbs6SM8t

alxmrs · 2026-07-23T21:44:53Z

@codex may I have your review once more?

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7eff9e31a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-23T21:50:51Z

+     - **Owner**: `Open-Athena`
+     - **Repository**: `Samudra`


Use the releasing repo's GitHub owner

When maintainers follow this trusted-publisher setup while the workflow is running in the repo advertised elsewhere in this tree (m2lines/Samudra in mkdocs.yml, installation, and contributing docs), the PyPI publisher entry will be registered for Open-Athena/Samudra instead of the repository that mints the OIDC token, so the first publish will fail authentication. Make this owner/repository, and the new package URLs, match the actual repo that runs .github/workflows/release.yml.

Useful? React with 👍 / 👎.

The canonical repo is m2lines/Samudra (Open-Athena/Samudra redirects there), so the OIDC repository claim at publish time is m2lines/Samudra. Update the trusted-publisher Owner in docs/releasing.md and the project URLs in pyproject.toml accordingly, so publishing resolves and metadata links are correct. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V9qeRacC2ZdJ4DCbs6SM8t

jder

Just a few questions from me but generally looks great, thanks! Does this also publish the config files? ie can you actually do uvx samudra train some/path/to/samudra_v2_om4.yaml somehow?

jder · 2026-07-24T16:49:08Z

+          elif [[ "${GITHUB_EVENT_NAME}" == "schedule" ]]; then
+            echo "mode=nightly" >> "$GITHUB_OUTPUT"
+            echo "input_version=" >> "$GITHUB_OUTPUT"
+          elif [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then


I was very confused how manual builds (ie workflow_dispatch) may or may not have a mode of manual. Should we maybe call that mode "smoke" or something rather than manual? Since you can manually trigger a stable or nightly build, too.

jder · 2026-07-24T16:50:03Z

+| Trigger | Mode | Version | Published? |
+| --- | --- | --- | --- |
+| Push a `v*` tag | `stable` | the tag, e.g. `v1.0.0` → `1.0.0` (setuptools-scm) | ✅ PyPI |
+| Daily `schedule` (06:00 UTC) | `nightly` | `<next-patch>.dev<YYYYMMDDhhmm>` | ✅ PyPI |


Do we really need/want a nightly build? We don't have that many changes in a day. Maybe weekly or something would be sufficient?

Weekly SGTM.

jder · 2026-07-24T16:51:35Z

+PyPI. To dry-run first, use **Actions → Release → Run workflow → mode: manual**;
+that builds and runs `twine check` without publishing.
+
+!!! note "Before the first tag"


Is this worth durably documenting?

Yeah, we can probably omit it.

jder · 2026-07-24T16:51:52Z

+    first tag, **`v0.0.1`**, makes the tag the single source of truth from then
+    on.
+
+## One-time trusted-publisher setup


Same question here -- should we just do this and delete these docs?

Yeah, that sounds fine to me, too.

jder · 2026-07-24T20:57:26Z

Should this be gated on successful tests? At least for the stable ones?

Responds to the PR #817 review feedback: - Gate publishing on tests. The publish job now `needs` a `test` job that calls test.yml (made reusable via `workflow_call`), so a red CPU suite blocks both stable and nightly uploads. A tag push doesn't otherwise run test.yml, hence invoking it from the release workflow. - Rename the build-only `manual` mode to `smoke`. workflow_dispatch is itself "manual," so the old name was ambiguous; `smoke` names what the build does (no-publish check), not how it's triggered. The local-version marker follows: `+manual.<sha>` -> `+smoke.<sha>`. - Move the scheduled dev release from daily to weekly (Mon 06:00 UTC); change volume doesn't warrant a daily nightly and the timestamp still keeps each build unique. - Add a `samudra` console script (`[project.scripts]` -> samudra.cli:main) dispatching `samudra train|eval|viz CONFIG ...` to the same entry points as `python -m samudra.train`, so installed users can run a task against their own config without a checkout. Packaging the example configs so the presets resolve by name is left as a documented follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V9qeRacC2ZdJ4DCbs6SM8t

alxmrs · 2026-07-27T18:51:53Z

Thanks @jder — all addressed in 8537600:

1. Gate publishing on tests ("at least for the stable ones"). Made test.yml reusable (workflow_call) and added a test job to the release workflow that publish now needs. A red CPU suite blocks the upload — for both stable and nightly. A tag push wouldn't otherwise run test.yml, which is why the release workflow calls it itself.

2. manual mode is confusingly named → smoke. Renamed the build-only mode to smoke throughout (workflow, package.py, docs), and the local-version marker with it (+manual.<sha> → +smoke.<sha>). It now names what the build does (a no-publish check) rather than how it starts — every mode can be hand-triggered via workflow_dispatch.

3. Nightly cadence → weekly. Cron is now 0 6 * * 1 (Mon 06:00 UTC). Agreed daily was overkill; the timestamp still keeps each dev build unique and PEP 440-ordered.

4. uvx samudra train some/config.yaml? Today: not quite — there was no console script, and configs/ lives at the repo root, so a wheel shipped only the library. I added a samudra console command (samudra train|eval|viz CONFIG ...) that forwards to the same entry points as python -m samudra.train, so an installed user can run a task against their own YAML without a checkout. Shipping the example presets so samudra train samudra_om4/train.yaml resolves by name needs the configs/ tree relocated under src/samudra/ (our src-layout means only files under the package land in the wheel) — that's a self-contained refactor I've left as a documented follow-up rather than fold into this packaging PR.

5 & 6. The "Before the first tag" note and the trusted-publisher setup docs — worth keeping? I kept both. The trusted-publisher steps are a genuine one-time task a maintainer has to do before the first publish can authorize (and are easy to get subtly wrong), so I'd rather they be written down. The "before the first tag" note is transient — happy to delete it the moment we cut v0.0.1, if you'd prefer it gone now I'll drop it.

alxmrs force-pushed the u/alxmrs/pypi branch from 6a6d84a to 24b8d05 Compare July 22, 2026 23:39

alxmrs commented Jul 22, 2026

View reviewed changes

Make room for initial version of 0.0.1

7e8e5d0

alxmrs force-pushed the u/alxmrs/pypi branch from b494d37 to 7e8e5d0 Compare July 22, 2026 23:48

alxmrs requested review from YuanYuan98 and jder July 22, 2026 23:49

YuanYuan98 approved these changes Jul 23, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 23, 2026

View reviewed changes

alxmrs and others added 2 commits July 23, 2026 15:36

Merge branch 'main' into u/alxmrs/pypi

e321dd3

jder approved these changes Jul 24, 2026

View reviewed changes

Uh oh!

Conversation

alxmrs commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Release channels

Versioning (setuptools-scm, hybrid)

Publish safety

CPU vs GPU

Security / hardening (per #766)

Also included

Verification

Follow-up required before first publish (maintainer, one-time)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alxmrs commented Jul 23, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

alxmrs commented Jul 23, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

jder left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jder commented Jul 24, 2026

Uh oh!

alxmrs commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alxmrs commented Jul 22, 2026 •

edited

Loading