Skip to content

WIP: Fix writing compound dtype datasets with a single field#1420

Draft
rly wants to merge 3 commits intodevfrom
fix/herd-shape-mismatch
Draft

WIP: Fix writing compound dtype datasets with a single field#1420
rly wants to merge 3 commits intodevfrom
fix/herd-shape-mismatch

Conversation

@rly
Copy link
Contributor

@rly rly commented Mar 6, 2026

Summary

  • Fix compound dtype detection in __list_fill__ to handle single-field compound dtypes (e.g., HERD's keys table with dtype [('key', 'O')])
  • The old check len(dtype) > 1 missed single-field compound dtypes, causing get_data_shape to return a 2-D shape that mismatched the 1-D maxshape, producing an h5py error
  • Use dtype.names is not None to correctly detect all compound dtypes

Fixes #1419

Test plan

  • test_write_dataset_list_single_field_compound_datatype — unit test for writing a single-field compound dtype dataset
  • test_write_herd_as_child — integration test writing/reading HERD as a child group of a container (mirrors the NWB use case)
  • Existing compound dtype and HERD tests continue to pass

🤖 Generated with Claude Code

The compound dtype check in __list_fill__ used `len(dtype) > 1`,
which missed single-field compound dtypes (e.g., HERD's keys table).
This caused the shape to be computed as 2-D via get_data_shape instead
of the correct 1-D `(len(data),)`, producing a shape/maxshape rank
mismatch error from h5py.

Use `dtype.names is not None` to correctly detect all compound dtypes.

Fixes #1419

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Mar 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.06%. Comparing base (0897b93) to head (790f249).

Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1420      +/-   ##
==========================================
+ Coverage   92.85%   93.06%   +0.21%     
==========================================
  Files          41       41              
  Lines        9989     9989              
  Branches     2054     2054              
==========================================
+ Hits         9275     9296      +21     
+ Misses        436      417      -19     
+ Partials      278      276       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Writing HERD as a child container produces shape/maxshape rank mismatch

1 participant