Fix Python 3.12 CI failures

[chrisspalm]

Summary

• Fix flaky test_create_cluster_validates_name assertion that raced between "Created" and "Pending" phases on Python 3.12 (relates to DaskCluster stuck in Pending state - Possible race condition? #968)
• Replace import pkg_resources with importlib.metadata.entry_points() in the controller for Python 3.12 compatibility (setuptools 82 removed pkg_resources)
• Pin setuptools<81 and jsonschema==4.17.3 in CI to fix k8s-crd-resolver / prance backend detection failure (openapi-spec-validator<0.5.0 still needs pkg_resources at import time)

Details

Test fix: The test previously asserted an exact phase ("Created"), but on Python 3.12 the async handler chain runs faster, causing a race where the poll catches "Pending" instead. Changed to assert != "Error" for valid names, which is correct across all Python versions.

Controller fix: pkg_resources was removed from setuptools 82. The controller used it for plugin discovery via iter_entry_points. Replaced with importlib.metadata.entry_points() (stdlib since Python 3.9).

CI fix: openapi-spec-validator 0.4.0 (required by k8s-crd-resolver) imports pkg_resources.resource_filename at module level. With setuptools 82, this import fails, causing prance to report "No validation backend available!". Pinning setuptools<81 restores pkg_resources availability. Also re-pins jsonschema==4.17.3 after dask[complete] installs, since openapi-spec-validator<0.5.0 needs jsonschema._legacy_validators (removed in 4.18+).

Test plan

• Reproduced prance backend detection failure locally with setuptools 82 + Python 3.12
• Verified fix locally: setuptools<81 restores openapi-spec-validator importability
• CI passes for Python 3.10, 3.11, 3.12

[chrisspalm]

Hey! Heads-up: I'm not deeply familiar with this codebase. I noticed the Python 3.12 CI failures and used an AI assistant to help diagnose and propose these fixes.

The analysis and local reproduction look solid, but I'd really appreciate a careful review since I can't vouch for every nuance of the controller/test behavior. In particular:

Test assertion change — switched from asserting exact phase "Created" to != "Error" for valid names, to avoid the async timing race between Created → Pending. Is this the right level of strictness?
importlib.metadata migration — replaced pkg_resources.iter_entry_points() with importlib.metadata.entry_points(). Should be equivalent but worth verifying.
CI setuptools<81 pin — workaround for setuptools 82 removing pkg_resources, which breaks openapi-spec-validator<0.5.0. Longer term, upgrading k8s-crd-resolver or its deps would be cleaner.
Happy to adjust anything based on feedback!

[jacobtomlinson]

I appreciate you taking a run at this, and especially for disclosing AI use here.

I don't think the fix for the flaky test is suitable though. I actually think identifying the source of the flakiness and referencing an issue is much more valuable. Modifying the test doesn't actually fix the source of the flakiness, it just masks it.

I think we should separate out the packaging changes to the flaky test changes. I don't mind merging this PR even with the flaky test failing. We should troubleshoot that separately.

[chrisspalm]

That makes sense — I've reverted the test change in b2c91da so this PR now only contains the packaging fixes

[chrisspalm]

@jacobtomlinson A little off topic here but I just read your post. I guess I almost fell into that category (Low effort AI created PR) here, but tried to keep the change minimal and easy to review. It would be great to get that Python 3.12 release.

Well worth a read!

[jacobtomlinson]

Thanks for the kind words!

I actually pointed a colleague to this PR as an example of a "Good AI PR". I think it was #971 (comment) that made the difference, you took the time to understand the goals and the changes and communicate that. You presented a solution but also asked open questions about what direction we should go in. You also came back and followed up, which is surprisingly rare for AI PRs!