Skip to content

Modernize packaging & tooling from the scverse template#215

Open
Zethson wants to merge 1 commit into
devfrom
modernize-packaging-tooling
Open

Modernize packaging & tooling from the scverse template#215
Zethson wants to merge 1 commit into
devfrom
modernize-packaging-tooling

Conversation

@Zethson

@Zethson Zethson commented Jun 11, 2026

Copy link
Copy Markdown
Member

What & why

gget still used a pre-PEP-517 stack (setup.py + setup.cfg + requirements.txt + dev-requirements.txt + MANIFEST.in, no in-repo linter config, no pre-commit). This PR adopts the good parts of the scverse cookiecutter template while keeping the mdbook docs, the bundled binaries/data, and the existing live-API test workflow working.

Packaging

  • Consolidated everything into a single pyproject.toml (hatchling backend, static version = 0.30.6). Verified the built wheel still bundles all 17 gget/bins/** binaries + gget/constants/** data and the gget console script; twine check --strict passes.
  • Python ≥ 3.12 ; classifiers + CI cover 3.12 / 3.13 / 3.14.
  • Runtime deps in [project]; test deps in a PEP 735 [dependency-groups] test group. cellxgene-census lives in a separate optional cellxgene group because it has no wheels for 3.14 yet (its tiledbsoma dep fails to build) — the gget cellxgene test now importorskips itself when the dep is absent, so the suite stays green on 3.14.

Linting / formatting

  • Added the full scverse .pre-commit-config.yaml (biome, pyproject-fmt, ruff lint+format, standard hygiene hooks) and the scverse ruff config (verbatim ruleset incl. pydocstyle).

CI / CD

  • Modernized ci.yml: uv, a pull_request trigger (PRs didn't run tests before), 3.12/3.13/3.14 matrix, Codecov upload, and an alls-green gate job — while preserving the scheduled run that saves & commits tests/pytest_results_py3.12.txt.
  • Added build.yml (package build check) and release.yml (PyPI trusted publishing).

Also

  • Fixes two latent bugs ruff surfaced: undefined Optional annotation in gget_cbio, and use-before-assignment of seqs in gget_muscle; plus a duplicate tearDown in the muscle tests.

⚠️ One manual follow-up

release.yml uses PyPI trusted publishing — it needs a one-time trusted publisher configured on PyPI for gget (workflow release.yml, environment pypi) before the first tagged release.

@codecov-commenter

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@Zethson Zethson force-pushed the modernize-packaging-tooling branch 2 times, most recently from 12a12dc to 6437c77 Compare June 11, 2026 20:17
@Zethson Zethson requested a review from lauraluebbert June 11, 2026 20:22
Comment thread .github/workflows/ci.yml
Comment on lines 123 to 192
@@ -112,22 +143,22 @@ jobs:
shell: bash
run: |
set -euo pipefail

BRANCH="${GITHUB_REF#refs/heads/}"
echo "Current branch: $BRANCH"

git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"

git add tests/pytest_results_py*.txt

if git diff --cached --quiet; then
echo "No changes to commit."
exit 0
fi

git commit -m "CI: update pytest results ($BRANCH)"

for attempt in 1 2 3 4 5; do
echo "Push attempt $attempt..."
git pull --rebase --autostash origin "$BRANCH" || true
@@ -136,7 +167,7 @@ jobs:
fi
sleep $((attempt * 5))
done

echo "Push failed after retries."
exit 1

@@ -159,3 +190,16 @@ jobs:
if [ "$code" != "0" ]; then
exit "$code"
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this, @lauraluebbert?
Do we even need this?

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
@Zethson Zethson force-pushed the modernize-packaging-tooling branch from 6437c77 to 9847d39 Compare June 12, 2026 07:01
Adopt the modern parts of the scverse cookiecutter while preserving gget's
mdbook docs, bundled binaries/data, and live-API test workflow.

Packaging:
- Consolidate setup.py + setup.cfg + requirements.txt + dev-requirements.txt
  + MANIFEST.in into a single pyproject.toml (hatchling backend, static
  version 0.30.6). Bundled binaries (gget/bins) and data (gget/constants)
  stay in the wheel; the `gget` console script is preserved.
- Require Python >= 3.12; classifiers and CI cover 3.12 / 3.13 / 3.14.
- Declare runtime deps in [project] and a [dependency-groups] `test` group;
  keep cellxgene-census in a separate optional `cellxgene` group (no wheels
  for newer Python yet) and have its test skip itself when the dep is absent.

Linting / formatting:
- Add the full scverse pre-commit config (biome, pyproject-fmt, ruff lint +
  format, standard hygiene hooks) plus the scverse ruff config.
- Run ruff format + fixes across the tree; add docstrings, `raise ... from`,
  isinstance checks, and targeted noqa to reach a green `prek run --all-files`.
- Vendored binaries, bundled data, test fixtures, and the CI report are
  excluded from formatting; Markdown hard line breaks are preserved.

CI:
- Modernize ci.yml (uv, pull_request trigger, 3.12/3.13/3.14 matrix, codecov,
  alls-green gate) while preserving the scheduled pytest-report commit-back.
- Add build.yml (package build check) and release.yml (PyPI trusted publishing).

Also fixes two latent bugs surfaced by ruff (undefined `Optional` in gget_cbio,
use-before-assignment of `seqs` in gget_muscle) and a duplicate tearDown in the
muscle tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Zethson Zethson force-pushed the modernize-packaging-tooling branch from 9847d39 to bc5b9c2 Compare June 12, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants