Skip to content

fix: pandas 3 compatibility#565

Merged
lewisjared merged 7 commits intomainfrom
pandas-3
Feb 24, 2026
Merged

fix: pandas 3 compatibility#565
lewisjared merged 7 commits intomainfrom
pandas-3

Conversation

@lewisjared
Copy link
Contributor

@lewisjared lewisjared commented Feb 23, 2026

Description

Resolve pandas FutureWarnings and compatibility issues to support both pandas 2 and 3:

  • constraints.py: Skip pd.concat when supplementary_group is empty; drop all-NA columns before concat to avoid dtype determination changes in pandas 3
  • example.py: Drop time_bnds before weighted mean (not after) to prevent dtype=object division error from resample+mean on datetime bounds; set explicit compat="override" on open_mfdataset
  • test_solve_helpers.py: Normalize None/NaT nulls in parquet round-trip assertions to handle stricter equality checks in pandas 3
  • Upgraded lockfile to the latest dependencies

Checklist

Please confirm that this pull request has done the following:

  • Tests added
  • Documentation added (where applicable)
  • Changelog item added to changelog/

@codecov
Copy link

codecov bot commented Feb 23, 2026

Codecov Report

❌ Patch coverage is 90.90909% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ate-ref-example/src/climate_ref_example/example.py 50.00% 0 Missing and 1 partial ⚠️
Flag Coverage Δ
core 93.22% <100.00%> (+0.03%) ⬆️
providers 89.56% <87.50%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...imate-ref-core/src/climate_ref_core/constraints.py 96.74% <100.00%> (+0.03%) ⬆️
...limate-ref-core/src/climate_ref_core/esgf/cmip7.py 93.80% <ø> (ø)
...ool/src/climate_ref_esmvaltool/diagnostics/base.py 95.38% <100.00%> (ø)
...limate-ref-ilamb/src/climate_ref_ilamb/standard.py 90.40% <100.00%> (+0.04%) ⬆️
...ate-ref-example/src/climate_ref_example/example.py 89.79% <50.00%> (-3.83%) ⬇️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

compat="override" conflicts with the default coords="different" in
xarray, causing a ValueError. The xarray compat warning is unrelated
to pandas 3 compatibility.
dependencies = [
"climate-ref-core",
"ilamb3>=2025.9.9",
"ilamb3>=2025.9.9,<2025.12", # to be relaxed in #548
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nocollier This will cause a merge conflict with #548, but we can resolve when merging

@lewisjared
Copy link
Contributor Author

Pre-req for #506

@lewisjared lewisjared merged commit dfe1cc8 into main Feb 24, 2026
26 checks passed
@lewisjared lewisjared deleted the pandas-3 branch February 24, 2026 03:52
lewisjared added a commit that referenced this pull request Feb 24, 2026
* origin/main: (80 commits)
  chore: Update comment
  chore: upgrade pins for ilamb
  fix: revert compat=override on open_mfdataset
  docs: add changelog for #565
  chore: Upgrade lockfile and fix some errors
  chore: add coverage
  chore: add default separator in alembic
  fix: time_coder warning
  chore: Pin to use tas
  fix(solver): preserve DataCatalog wrapper in apply_dataset_filters
  fix(tests): use to_frame() when accessing DataCatalog in solver tests
  docs: Changelog
  chore: run the finalise in threads
  chore: clean up
  chore: add fix changelog entry for PR #561
  feat(cli): add --dataset-filter option to datasets list command
  chore: add changelog entry for PR #561
  feat(solver): add --dataset-filter option to filter input datasets when solving
  chore: Support env variables for parser
  feat(solver): add optional --limit flag to solve command
  ...
lewisjared added a commit that referenced this pull request Feb 25, 2026
* origin/main: (86 commits)
  Bump version: 0.11.0 → 0.11.1
  docs: add changelog for #567
  fix: reduce memory during ingestion and add solve logging
  fix: prevent DRS re-ingestion from regressing finalised datasets
  Bump version: 0.10.0 → 0.11.0
  chore: Update comment
  chore: upgrade pins for ilamb
  fix: revert compat=override on open_mfdataset
  docs: add changelog for #565
  chore: Upgrade lockfile and fix some errors
  chore: add coverage
  chore: add default separator in alembic
  fix: time_coder warning
  chore: Pin to use tas
  fix(solver): preserve DataCatalog wrapper in apply_dataset_filters
  fix(tests): use to_frame() when accessing DataCatalog in solver tests
  docs: Changelog
  chore: run the finalise in threads
  chore: clean up
  chore: add fix changelog entry for PR #561
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant