Skip to content

Freethreading#215

Open
ColmTalbot wants to merge 13 commits intoliberfa:mainfrom
ColmTalbot:freethreading
Open

Freethreading#215
ColmTalbot wants to merge 13 commits intoliberfa:mainfrom
ColmTalbot:freethreading

Conversation

@ColmTalbot
Copy link
Copy Markdown

The changes I needed were

  • making erfa leap seconds thread safe
  • making the leap second expiration global variables thread local
  • I tested the impure aper functions, but those seem to not actually modify the input in place in pyerfa, so those are safe as far as I can tell.

I ran all of the tests with pytest-run-parallel and they pass (except the docstring test, which pytest-run-parallel can't handle).

Remaining questions I have:

  • should I write something up in the docs somewhere?
  • I added a py314t test to the CI suite, is that alright? Should I add more tests?

See #209 cc @neutrinoceros @mhvk

Needs liberfa/erfa#108.

@neutrinoceros
Copy link
Copy Markdown
Contributor

I added a py314t test to the CI suite, is that alright?

I don't think it's sufficient in itself. We should probably ensure we set PYTHON_GIL=0 in the corresponding tox env for it to mean anything.

Should I add more tests?

it would indeed be wise to add at least one test in the style described in https://py-free-threading.github.io/testing/
Ideally, build this test around the one known concurrency issue (leap seconds table updates), give it its own module for simplicity of selection, and extend test-command under [tool.cibuildwheel] to run this one module a couple hundred times in pytest to give it a solid chance to capture race conditions somewhat reliably.
It'd be even better to have all of this on your branch before the fix is applied to liberfa, so we can easily verify that the test goes from failed to passed

@ColmTalbot
Copy link
Copy Markdown
Author

ColmTalbot commented Mar 31, 2026

I pushed to a test branch with just the CI changes to verify that the existing versions fails, see the logs here. The results are below, the four failures are due to setting the global state in erfa/pyerfa.

Currently, all of the tests will run with 100 threads, it still only takes a few seconds (it isn't the slowest job), so hopefully this is fine. There's a passing log here.

============================= test session starts ==============================
platform linux -- Python 3.14.3, pytest-9.0.2, pluggy-1.6.0
cachedir: .tox/test-freethreaded/.pytest_cache
rootdir: /home/runner/work/pyerfa/pyerfa
configfile: pyproject.toml
plugins: run-parallel-0.8.2, doctestplus-1.7.1
collected 281 items
Collected 281 items to run in parallel
../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py · [  0%]
·······················e···eees·                                         [ 11%]
../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_ufunc.py · [ 12%]
········································································ [ 37%]
········································································ [ 63%]
········································································ [ 88%]
·······························                                          [100%]
==================================== ERRORS ====================================
____________ ERROR at call of TestLeapSeconds.test_set_leap_seconds ____________
self = <erfa.tests.test_erfa.TestLeapSeconds object at 0x30cd995ce50>
    def test_set_leap_seconds(self):
>       assert erfa.dat(2018, 1, 1, 0.) == 37.0
E       assert np.float64(36.0) == 37.0
E        +  where np.float64(36.0) = <function dat at 0x30cda7fe7c0>(2018, 1, 1, 0.0)
E        +    where <function dat at 0x30cda7fe7c0> = erfa.dat
../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py:456: AssertionError
__________ ERROR at call of TestLeapSeconds.test_update_leap_seconds ___________
self = <erfa.tests.test_erfa.TestLeapSeconds object at 0x30cd995d710>
    def test_update_leap_seconds(self):
        assert erfa.dat(2018, 1, 1, 0.) == 37.0
        leap_seconds = erfa.leap_seconds.get()
        # Get old and new leap seconds
        old_leap_seconds = leap_seconds[leap_seconds['year'] < 2017]
        new_leap_seconds = leap_seconds[leap_seconds['year'] >= 2017]
        # Updating with either of these should do nothing.
        n_update = erfa.leap_seconds.update(new_leap_seconds)
>       assert n_update == 0
E       assert 1 == 0
../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py:495: AssertionError
_______ ERROR at call of TestLeapSeconds.test_with_expiration[datetime] ________
self = <erfa.tests.test_erfa.TestLeapSeconds object at 0x30cd995dd50>
expiration = datetime.datetime(2345, 1, 1, 0, 0)
    @pytest.mark.parametrize(
        "expiration", [datetime(2345, 1, 1), "1 January 2345", astropy_time], ids=type
    )
    def test_with_expiration(self, expiration):
        class ExpiringArray(np.ndarray):
            expires = expiration
    
        leap_seconds = erfa.leap_seconds.get()
        erfa.leap_seconds.set(leap_seconds.view(ExpiringArray))
        assert erfa.leap_seconds.expires == datetime(2345, 1, 1)
    
        # Get old and new leap seconds
        old_leap_seconds = leap_seconds[:-10]
        new_leap_seconds = leap_seconds[-10:]
    
        erfa.leap_seconds.set(old_leap_seconds)
        # Check expiration is reset
        assert erfa.leap_seconds.expires != datetime(2345, 1, 1)
        # Update with missing leap seconds.
        n_update = erfa.leap_seconds.update(
            new_leap_seconds.view(ExpiringArray))
>       assert n_update == len(new_leap_seconds)
E       AssertionError: assert 0 == 10
E        +  where 10 = len(array([(1993, 7, 28.), (1994, 7, 29.), (1996, 1, 30.), (1997, 7, 31.),\n       (1999, 1, 32.), (2006, 1, 33.), (2009, 1...['year', 'month', 'tai_utc'], 'formats': ['<i4', '<i4', '<f8'], 'offsets': [0, 4, 8], 'itemsize': 16, 'aligned': True}))
../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py:537: AssertionError
__________ ERROR at call of TestLeapSeconds.test_with_expiration[str] __________
self = <erfa.tests.test_erfa.TestLeapSeconds object at 0x30cd8738410>
expiration = '1 January 2345'
    @pytest.mark.parametrize(
        "expiration", [datetime(2345, 1, 1), "1 January 2345", astropy_time], ids=type
    )
    def test_with_expiration(self, expiration):
        class ExpiringArray(np.ndarray):
            expires = expiration
    
        leap_seconds = erfa.leap_seconds.get()
        erfa.leap_seconds.set(leap_seconds.view(ExpiringArray))
        assert erfa.leap_seconds.expires == datetime(2345, 1, 1)
    
        # Get old and new leap seconds
        old_leap_seconds = leap_seconds[:-10]
        new_leap_seconds = leap_seconds[-10:]
    
        erfa.leap_seconds.set(old_leap_seconds)
        # Check expiration is reset
        assert erfa.leap_seconds.expires != datetime(2345, 1, 1)
        # Update with missing leap seconds.
        n_update = erfa.leap_seconds.update(
            new_leap_seconds.view(ExpiringArray))
>       assert n_update == len(new_leap_seconds)
E       AssertionError: assert 0 == 10
E        +  where 10 = len(array([(1993, 7, 28.), (1994, 7, 29.), (1996, 1, 30.), (1997, 7, 31.),\n       (1999, 1, 32.), (2006, 1, 33.), (2009, 1...['year', 'month', 'tai_utc'], 'formats': ['<i4', '<i4', '<f8'], 'offsets': [0, 4, 8], 'itemsize': 16, 'aligned': True}))
../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py:537: AssertionError
************************** pytest-run-parallel report **************************
All tests were run in parallel! 🎉
=========================== short test summary info ============================
PARALLEL FAILED ../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py::TestLeapSeconds::test_set_leap_seconds - assert np.float64(36.0) == 37.0
 +  where np.float64(36.0) = <function dat at 0x30cda7fe7c0>(2018, 1, 1, 0.0)
 +    where <function dat at 0x30cda7fe7c0> = erfa.dat
PARALLEL FAILED ../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py::TestLeapSeconds::test_update_leap_seconds - assert 1 == 0
PARALLEL FAILED ../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py::TestLeapSeconds::test_with_expiration[datetime] - AssertionError: assert 0 == 10
 +  where 10 = len(array([(1993, 7, 28.), (1994, 7, 29.), (1996, 1, 30.), (1997, 7, 31.),\n       (1999, 1, 32.), (2006, 1, 33.), (2009, 1...['year', 'month', 'tai_utc'], 'formats': ['<i4', '<i4', '<f8'], 'offsets': [0, 4, 8], 'itemsize': 16, 'aligned': True}))
PARALLEL FAILED ../../.tox/test-freethreaded/lib/python3.14t/site-packages/erfa/tests/test_erfa.py::TestLeapSeconds::test_with_expiration[str] - AssertionError: assert 0 == 10
 +  where 10 = len(array([(1993, 7, 28.), (1994, 7, 29.), (1996, 1, 30.), (1997, 7, 31.),\n       (1999, 1, 32.), (2006, 1, 33.), (2009, 1...['year', 'month', 'tai_utc'], 'formats': ['<i4', '<i4', '<f8'], 'offsets': [0, 4, 8], 'itemsize': 16, 'aligned': True}))
================== 276 passed, 1 skipped, 4 errors in 19.65s ===================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants