Skip to content

ilamb3 updates #548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nocollier wants to merge 15 commits into
base: main
Choose a base branch
from
Open

ilamb3 updates #548

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
2deee16
Temporarily remove the ocean heat content diagnostic as per email fro…
nocollier Feb 18, 2026
2a6f5c8
changes to configure files, data references updated where possible
nocollier Feb 19, 2026
ade4287
change how models are named in the plots and output filenames
nocollier Feb 19, 2026
c77383e
append the obs4REF dataset registry to the ilamb so these keys can be…
nocollier Feb 19, 2026
55553f8
add the evspsbl-pr metric only over land as requested
nocollier Feb 19, 2026
fe0b9f9
add ilamb model color selector, only 1 model but at least won't be black
nocollier Feb 19, 2026
d6d3b9c
bump ilamb3 version
nocollier Feb 19, 2026
2cf846b
changelog
nocollier Feb 19, 2026
3cc6603
fix: lockfile
lewisjared Feb 21, 2026
67d0378
expand ilamb3 ref configure files to allow for dictionaries with data…
nocollier Feb 23, 2026
ef7d6ad
Merge remote-tracking branch 'origin/main' into ilamb-updates
lewisjared Feb 24, 2026
8815a50
chore: fix mypy
lewisjared Feb 24, 2026
5393d00
chore: remove ohc dataset
lewisjared Feb 24, 2026
e8fab3e
chore: loosen facet filter type
lewisjared Feb 24, 2026
e9ad222
chore: clean up datarequirements
lewisjared Feb 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/548.feature.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update `ilamb3`, remove the `ohc-noaa` diagnostic and add the `evspsbl-pr` diagnostic.
4 changes: 2 additions & 2 deletions packages/climate-ref-core/src/climate_ref_core/datasets.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"""

import hashlib
from collections.abc import Collection, Iterable, Iterator
from collections.abc import Collection, Iterable, Iterator, Mapping
from typing import Any

import pandas as pd
Expand All @@ -15,7 +15,7 @@
__all__ = ["Selector", "SourceDatasetType"]


def _clean_facets(raw_values: dict[str, str | Collection[str]]) -> dict[str, tuple[str, ...]]:
def _clean_facets(raw_values: Mapping[str, str | Collection[str]]) -> dict[str, tuple[str, ...]]:
"""
Clean the value of a facet filter to a tuple of strings
"""
Expand Down
2 changes: 1 addition & 1 deletion packages/climate-ref-ilamb/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ classifiers = [
]
dependencies = [
"climate-ref-core",
"ilamb3>=2025.9.9,<2025.12", # to be relaxed in #548
"ilamb3>=2026.2.19",
"scipy<1.16", # https://github.com/statsmodels/statsmodels/issues/9584
"xarray<2025.11", # ilamb3 incompatibility with integrate_space units handling
]
Expand Down
39 changes: 30 additions & 9 deletions packages/climate-ref-ilamb/src/climate_ref_ilamb/configure/ilamb.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@ registry: ilamb

gpp-WECANN:
sources:
# TODO: Update to use the obs4REF equiv
gpp: ilamb/gpp/WECANN/gpp.nc
gpp: obs4REF/ColumbiaU/WECANN-1-0/mon/gpp/gn/20250516/gpp_mon_WECANN-1-0_REF_gn_200701-201512.nc
relationships:
pr: ilamb/pr/GPCPv2.3/pr.nc
pr: obs4REF/NOAA-NCEI/GPCP-2-3/mon/pr/gn/v20210727/pr_mon_GPCP-2-3_PCMDI_gn_197901-201907.nc
tas: ilamb/tas/CRU4.02/tas.nc
variable_cmap: Greens

Expand All @@ -16,8 +15,9 @@ gpp-FLUXNET2015:

mrro-LORA:
sources:
# TODO: Update to use the obs4REF equiv
mrro: ilamb/mrro/LORA/LORA.nc
mrro:
obs_source: obs4ref
source_id: LORA-1-1
variable_cmap: Blues

mrsos-WangMao:
Expand All @@ -36,17 +36,24 @@ cSoil-HWSD2:
cSoil: ilamb/cSoil/HWSD2/cSoil_fx_HWSD2_19600101-20220101.nc
variable_cmap: viridis

lai-AVH15C1:
lai-NOAA-NCEI-LAI-AVHRR-5-0:
sources:
lai: ilamb/lai/AVH15C1/lai.nc
lai:
obs_source: obs4mips
source_id: NOAA-NCEI-LAI-AVHRR-5-0
variable_cmap: Greens

nbp-Hoffman:
analyses:
- nbp
sources:
# TODO: Update to use the obs4REF equiv
nbp: ilamb/nbp/HOFFMAN/nbp_1850-2010.nc
nbp:
obs_source: obs4ref
variable_id: nbp
source_id: Hoffman
transforms:
- integrate_space:
varname: nbp

snc-ESACCI:
sources:
Expand All @@ -58,3 +65,17 @@ burntFractionAll-GFED:
alternate_vars:
- burntFractionAll
test_source_id: CESM2

emp-GLEAMGPCP2.3:
analysis_variable: emp
sources:
et: ilamb/evspsbl/GLEAMv3.3a/et.nc
pr: obs4REF/NOAA-NCEI/GPCP-2-3/mon/pr/gn/v20210727/pr_mon_GPCP-2-3_PCMDI_gn_197901-201907.nc
transforms:
- select_time:
vmin: 2000-01
vmax: 2015-01
- expression:
expr: emp = et - pr
alternate_vars:
- evspsbl
21 changes: 3 additions & 18 deletions packages/climate-ref-ilamb/src/climate_ref_ilamb/configure/iomb.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ registry: iomb

thetao-WOA2023-surface:
sources:
# TODO: Update to use the obs4REF equiv
# NOTE: obs4REF equiv starts beyond historical, cannot use for CMIP6
Copy link
Contributor

@lewisjared lewisjared Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Darwyn72 Can you please take a look at this and the other datasets? Is this due to using a newer version?

Copy link

@Darwyn72 Darwyn72 Feb 20, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lewisjared this is the first time I've seen this repo, but if you are reading this data from the CEDA Archive there is a newer version of this dataset prepared by Morgan Steckler at ORNL using ILAMB. It's in the CEDA Archive under NOAA-NCEI-OCL/mon/thetao/v20251024 - the previous version was under v20250923. If you are not reading it from CEDA Archive. This dataset only provides climatological means, so the start and end time are not applicable, the climatological mean is calculated over the following two time periods: 2005 – 2014, 2015 – 2022

I could send a copy of this CMOR-like dataset to you?

Copy link
Contributor

@lewisjared lewisjared Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes please

Copy link

@Darwyn72 Darwyn72 Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lewisjared This is the latest list of ILAMB prepared datasets. Morgan Steckler asks that any errors found by the REF with the data are logged here I can't forward the data to you by email as the files are too large. For WOA2023 the two files for thetao and so are dated 2025-10-24. Thanks, Paul

Copy link
Contributor

@lewisjared lewisjared Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this be the official location of these data? Will these be moved to CEDA?

I'm a bit wary as I'm pretty sure the REF isn't using all of the latests datasets you have curated. Is there a complete list of download URLs for all datasets that have been vetted?

Copy link

@Darwyn72 Darwyn72 Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lewisjared these are the datasets I have put on the CEDA Archive - although this is not (yet) public. Alison Waterfall at CEDA is supposed to put these data onto ESGF-NG once it is ready, either in an obs4REF or obs4MIPs project, or some temporary folder for the REF.
image
Ranjini has an Airtable that mirrors Table A1 and B1 in the REF paper for reference. This is a link to a cutdown version of the Master view. Maybe this is helpful?

thetao: ilamb/WOA/thetao_mon_WOA_A5B4_gn_200501-201412.nc
variable_cmap: Reds
transforms:
Expand All @@ -13,7 +13,7 @@ thetao-WOA2023-surface:

so-WOA2023-surface:
sources:
# TODO: Update to use the obs4REF equiv
# NOTE: obs4REF equiv starts beyond historical, cannot use for CMIP6
so: ilamb/WOA/so_mon_WOA_A5B4_gn_200501-201412.nc
transforms:
- select_depth:
Expand All @@ -30,20 +30,5 @@ amoc-RAPID:
transforms:
- msftmz_to_rapid
sources:
# TODO: Update to use the obs4REF equiv
# obs4REF equiv does not work
amoc: ilamb/RAPID/amoc_mon_RAPID_BE_NA_200404-202302.nc

ohc-NOAA:
sources:
ohc: ilamb/NOAA/ohc_yr_OHC_BE_gm_200506-202406.nc
related_vars:
- thetao
- volcello
transforms:
- select_depth:
vmin: 0
vmax: 2000
- ocean_heat_content:
reference_year: 2005
analyses:
- accumulate
1 change: 1 addition & 0 deletions packages/climate-ref-ilamb/src/climate_ref_ilamb/dataset_registry/ilamb.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ ilamb/tas/CRU4.02/tas.nc sha1:2674da18a1a93483b50b1626e7a7ab741bf53d09
ilamb/nbp/HOFFMAN/nbp_1850-2010.nc sha1:8350af00614d6afc6b70ad314aa499a9ece80ec2
ilamb/snc/Snow-cci/snc_mon_Snow-cci_BE_gn_198201-201906.nc sha1:c0bfecd2f8b886e9301428d28bb6ff0507601be2
ilamb/burntFractionAll/GFED/burntArea.nc sha1:cf9d73c6a8bfc594737c9ba6ca4df613df4a28ab
ilamb/evspsbl/GLEAMv3.3a/et.nc sha1:5aaf73949af8c6b509ef16f684aa8efeccd983e2
1 change: 0 additions & 1 deletion packages/climate-ref-ilamb/src/climate_ref_ilamb/dataset_registry/iomb.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
ilamb/WOA/so_mon_WOA_A5B4_gn_200501-201412.nc sha1:831c42c3b2ba443c255150289a2c725d7f3e5838
ilamb/WOA/thetao_mon_WOA_A5B4_gn_200501-201412.nc sha1:86d9056208291d76233e65b26c658c1fa54c3ea6
ilamb/RAPID/amoc_mon_RAPID_BE_NA_200404-202302.nc sha1:3efe773e5c2a3c832977791ff7fd9cb9f473fe65
ilamb/NOAA/ohc_yr_OHC_BE_gm_200506-202406.nc sha1:a918799d8e24e4f0015b9047a74d470ae9f0445c
Loading
Loading