Skip to content

Comments

Remove the use of out_name #547

Open
lewisjared wants to merge 10 commits intomainfrom
remove-out-name
Open

Remove the use of out_name #547
lewisjared wants to merge 10 commits intomainfrom
remove-out-name

Conversation

@lewisjared
Copy link
Contributor

@lewisjared lewisjared commented Feb 18, 2026

Description

  • Simplified the extraction of variable mappings from the CMIP7 Data Request
  • Removed out_name usages
  • Don't fallback to defaults when defining filenames and drs

Checklist

Please confirm that this pull request has done the following:

  • Tests added
  • Documentation added (where applicable)
  • Changelog item added to changelog/

- Simplified the extraction of variable mappings from the CMIP7 Data Request
- Removed out_name usages
@codecov
Copy link

codecov bot commented Feb 18, 2026

Codecov Report

❌ Patch coverage is 93.93939% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...te-ref-core/src/climate_ref_core/cmip6_to_cmip7.py 93.22% 4 Missing ⚠️
Flag Coverage Δ
core 93.20% <93.93%> (+0.04%) ⬆️
providers 89.93% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...limate-ref-core/src/climate_ref_core/esgf/cmip7.py 93.85% <100.00%> (+0.05%) ⬆️
...ages/climate-ref/src/climate_ref/datasets/cmip7.py 85.71% <ø> (ø)
...ages/climate-ref/src/climate_ref/models/dataset.py 98.93% <ø> (ø)
...te-ref-core/src/climate_ref_core/cmip6_to_cmip7.py 94.50% <93.22%> (+1.57%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lewisjared
Copy link
Contributor Author

@bouweandela this will cause duplicate test data now following all of the DRS /.cache/climate-ref can be deleted as the converted files now also use the common cache dir (/.cache/climate_ref)

attrs["realization_index"] = convert_variant_index(attrs.get("realization_index", 1), "r")
attrs["initialization_index"] = convert_variant_index(attrs.get("initialization_index", 1), "i")
attrs["physics_index"] = convert_variant_index(attrs.get("physics_index", 1), "p")
attrs["forcing_index"] = convert_variant_index(attrs.get("forcing_index", 1), "f")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow my data always seems to end up with f1, even when the input data has f2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fall back to using value parsed from variant label, but it could also because convert_variant_index explicitly checks for int so numpy ints fail. Either way its more defensive now

Copy link
Contributor

@bouweandela bouweandela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I compared the results with what CMOR generates. It is fairly similar, except for the issue with the name of the primary variable.

I also noticed that the output from the REF contains a scalar height coordinate on the bounds variables, which does not look right. For example:

netcdf tas_tmaxavg-h2m-hxy-u_mon_glb_gn_UKESM1-0-LL_historical_r1i1p1f2_185001-194912 {
dimensions:
	time = UNLIMITED ; // (1200 currently)
	bnds = 2 ;
	lat = 144 ;
	lon = 192 ;
variables:
	double time_bnds(time, bnds) ;
		time_bnds:_FillValue = NaN ;
		time_bnds:coordinates = "height" ;
...
	double height ;
		height:_FillValue = NaN ;
		height:units = "m" ;
		height:axis = "Z" ;
		height:positive = "up" ;
		height:long_name = "height" ;
		height:standard_name = "height" ;

note the line time_bnds:coordinates = "height" ;

Two bugs caused non-r1i1p1f1 datasets to always produce r1i1p1f1:

1. convert_variant_index() used isinstance(value, int) which missed
   numpy integer types (np.int32, np.int64) from netCDF attributes,
   falling through to the default return of f"{prefix}1"

2. convert_cmip6_to_cmip7_attrs() defaulted all missing individual
   indexes to 1 and rebuilt variant_label from those defaults,
   overwriting the correct variant_label even when it was present

Fix: parse variant_label as fallback when individual indexes are
missing, and handle numpy integer types in convert_variant_index().
@lewisjared
Copy link
Contributor Author

I learnt something about xarray today with that spurious bound!

* origin/main: (86 commits)
  Bump version: 0.11.0 → 0.11.1
  docs: add changelog for #567
  fix: reduce memory during ingestion and add solve logging
  fix: prevent DRS re-ingestion from regressing finalised datasets
  Bump version: 0.10.0 → 0.11.0
  chore: Update comment
  chore: upgrade pins for ilamb
  fix: revert compat=override on open_mfdataset
  docs: add changelog for #565
  chore: Upgrade lockfile and fix some errors
  chore: add coverage
  chore: add default separator in alembic
  fix: time_coder warning
  chore: Pin to use tas
  fix(solver): preserve DataCatalog wrapper in apply_dataset_filters
  fix(tests): use to_frame() when accessing DataCatalog in solver tests
  docs: Changelog
  chore: run the finalise in threads
  chore: clean up
  chore: add fix changelog entry for PR #561
  ...
Copy link
Contributor

@bouweandela bouweandela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants