Set dandischema dependency to 0.12.0 and beyond#1744
Merged
yarikoptic merged 2 commits intomasterfrom Nov 24, 2025
Merged
Conversation
25 tasks
yarikoptic
reviewed
Nov 20, 2025
Member
|
0.12.0 schema was released |
Member
|
Note: as server does not really need anything "new" from dandi-cli, we ideally have server released/deployed first (people would be able to upload just fine) and then we ensure it is all green here and then merge/release |
20 tasks
This allows syncing of `dandischema` version with `dandi-archive` to resolve the `dandischema` versioning issue stated in dandi/dandi-schema#344 as part of the effort to address dandi/dandi-schema#299
…requirement Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
yarikoptic
force-pushed
the
sync-schema-version-with-archive
branch
from
7386800 to
f81c32e
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1744 +/- ##
=======================================
Coverage 75.08% 75.08%
=======================================
Files 84 84
Lines 11874 11874
=======================================
Hits 8915 8915
Misses 2959 2959
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
1 task
yarikoptic
approved these changes
Nov 24, 2025
|
🚀 PR was released in |
yarikoptic
added a commit
that referenced
this pull request
Feb 13, 2026
The dandi-archive CLI integration tests started hanging for 6 hours on November 24, 2025 (dandi/dandi-archive#1762). Investigation of tinuous CI logs showed: - Nov 21 (last success): test_nwb2asset_remote_asset XFAIL'd in 0.4s - Nov 24 (first hang): same test hung for 6 hours until GitHub killed it - ALL test runs on Nov 24-25 hung, across unrelated PRs The code flow that hangs is: nwb2asset() → get_metadata() → _get_pynwb_metadata() → open_readable() → RemoteReadableAsset.open() → fsspec.open(url).open() → aiohttp HTTP read from minio in Docker → fsspec sync() blocks in threading.Event.wait() The key environmental change between Nov 21 and Nov 24 was dandi-cli PR #1744 updating dandischema from <0.12.0 to ~=0.12.0. With dandischema 0.11.x, the test hit a quick model validation mismatch (completing as XFAIL in 0.4s before reaching the fsspec read). With dandischema 0.12.0 (vendor-configurable models, schema 0.7.0), that mismatch no longer occurs, so the test now proceeds to the actual fsspec HTTP read — which hangs. The hang itself is a known interaction between h5py, fsspec, and GC: - h5py holds a global lock while reading from Python file objects - fsspec's sync() runs async aiohttp coroutines on a background thread and blocks the calling thread in threading.Event.wait() - Without socket-level timeouts, aiohttp blocks forever on stalled connections (aio-libs/aiohttp#11740) - GC running during this window can deadlock with h5py's lock (h5py/h5py#2019) The fix: pass explicit ClientTimeout to aiohttp via fsspec's client_kwargs so that stalled connections raise TimeoutError instead of blocking indefinitely. Additionally, the dandi-archive CI never had a pytest --timeout because dandi-cli's tox.ini [pytest] addopts (--timeout=300) are not read when pytest runs from the dandi-archive rootdir via `pytest --pyargs dandi`. References: - fsspec/filesystem_spec#1666 - h5py/h5py#2019 - aio-libs/aiohttp#11740 - #1762 - #1450 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See individual commit message for details.
The CIs can only pass only if