Skip to content

fix: remove unsupported dd.DataFrame type hint in parquet_reader#1774

Merged
SFJohnson24 merged 2 commits into
cdisc-org:mainfrom
filippsatverily:filipps/pandas3-fix-parquet-reader-type
Jun 22, 2026
Merged

fix: remove unsupported dd.DataFrame type hint in parquet_reader#1774
SFJohnson24 merged 2 commits into
cdisc-org:mainfrom
filippsatverily:filipps/pandas3-fix-parquet-reader-type

Conversation

@filippsatverily

Copy link
Copy Markdown
Contributor

Removes the Union[pd.DataFrame, dd.DataFrame] type hint from _format_floats in parquet_reader. This method calls .map() which is only available on pd.DataFrame — it was never called with a dask DataFrame. The Union type hint pulls in dd.DataFrame which pandas 3.0 doesn't support in the same way, and the unused branch masks a type error.

Tested scenarios:

  • Full pytest suite: 1746 passed, 11 skipped, 0 failed (pandas 2.3.3, dask 2025.12.0)
  • Ran validation on CDISC_Pilot_Study_v4_FIXED.json: 201 SUCCESS, 6 SKIPPED, 0 errors

@filippsatverily filippsatverily marked this pull request as ready for review June 22, 2026 19:44
@filippsatverily

Copy link
Copy Markdown
Contributor Author

@SFJohnson24 see next commit for pandas 3 upgrade from #1745

@SFJohnson24

Copy link
Copy Markdown
Collaborator

passes test suite: see #1775, high level review, pilot study test.

@SFJohnson24 SFJohnson24 self-requested a review June 22, 2026 20:55
@SFJohnson24 SFJohnson24 merged commit c6465e7 into cdisc-org:main Jun 22, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants