Drop Python 3.10?

[crusaderky]

Python 3.10 support has become unsustainable in the current dask and distributed CI. Because so many packages no longer support it as of the latest version, conda environment resolution now takes in excess of 30 minutes, and any changes to the environment file end up with CI hanging completely.

I can see two ways out (one does not exclude the other):

• Drop Python 3.10. Please speak up here if dropping Python 3.10 would affect you.
• Migrate to a lockfile-based package manager. Personally, I've been aggressively using pixi for the last year and I'm in love with it.

[crusaderky]

CC @dask/maintenance

[martindurant]

Is resolution a problem in pip too? Python 3.10 itself is maintained until later this year.

[crusaderky]

Is resolution a problem in pip too?

I don't think so (untested). However that would mean having one env managed by pip and the others by conda. It doesn't seem too healthy?

Python 3.10 itself is maintained until later this year

Does it matter though? It exited SPEC 0 years ago.

[martindurant]

Conda stats for april show:

pkg_python
3.6      6683
2.7      7830
3.10     9246
3.12    13839
3.11    14850
        16975
3.13    19203

This is for the anaconda main channel, not the more common conda-forge, because only noarch exists on the latter (which is the blank entry, above).

[martindurant]

Does it matter though? It exited SPEC 0 years ago.

Yes and no! It still being an officially supported version mean that people don't feel pressured to move on if they have existing working environments with it.

[jameslamb]

Hope you don't mind the drive-by comment, I'm still subscribed here from a time in the past when I was more active with Dask 😊

Migrate to a lockfile-based package manager.

For what it's worth, in the LightGBM project we've been able to keep our conda solve times manageable for end-of-life and mostly-no-longer-supported Python versions by just recording the package versions from a successful solve and always reusing that in CI. Example:

• conda: https://github.com/lightgbm-org/LightGBM/blob/2a675f269d08fd51fad23d67a0214f574acb384e/.ci/conda-envs/ci-core-py39.txt
• pip: https://github.com/lightgbm-org/LightGBM/blob/2a675f269d08fd51fad23d67a0214f574acb384e/.ci/pip-envs/requirements-oldest.txt

That's a lighter-weight approach than generating a full lockfile and it's a weaker test guarantee (just "there's at least 1 solvable environment with this Python version, and LightGBM's Python code is still functional when using that Python version") but we've been using that approach for a few years and it's worked pretty well so far.