fix: apply version filtering and fix supplementary dataset selection in regression analysis

[lewisjared]

Description

Fixes two issues in the dataset selection logic that caused spurious entries in regression analysis results:

1. Version filtering in dataset queries (datasets/base.py): Added proper version filtering to query_datasets and query_facets methods. Previously, version constraints were silently ignored, causing older dataset versions to leak into results. The version filter supports both exact matching and ordering comparisons (e.g. v20190731 vs v20200101).

2. Supplementary dataset selection (constraints.py): Fixed AddSupplementaryDataset to pick a single best-matching dataset per group instead of accumulating multiple candidates across score ties. Previously, when multiple supplementary datasets tied on matching score, all were selected, leading to spurious experiment entries (e.g. 1pctCO2, esm-1pct-brch-1000PgC) appearing alongside expected historical and SSP data.

These fixes together eliminate unexpected experiment/version combinations from regression outputs.

Checklist

Please confirm that this pull request has done the following:

• Tests added
• Documentation added (where applicable)
• Changelog item added to changelog/

[lewisjared]

@bouweandela what do you think about 1cb40c0 ?

[lewisjared]

Closes #545 and #543

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
core	92.51% <100.00%> (+0.04%)	⬆️
providers	89.65% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...imate-ref-core/src/climate_ref_core/constraints.py	96.75% <100.00%> (+0.04%)	⬆️
...kages/climate-ref/src/climate_ref/datasets/base.py	98.47% <100.00%> (+0.04%)	⬆️
...kages/climate-ref/src/climate_ref/solve_helpers.py	98.92% <100.00%> (+0.02%)	⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lewisjared]

Need to check ps which is another potential supplementary. volcello might be time-dependent.

[bouweandela]

For reference, here is an overview of the supplementary variables used by ESMValCore preprocessor functions: https://docs.esmvaltool.org/projects/ESMValCore/en/latest/recipe/preprocessor.html#supplementary-variables-ancillary-variables-and-cell-measures. Not all preprocessor functions are used by the REF, so not all of them may be needed.

Note that I have also used the AddSupplementary constraint to add the historical experiment to the scenarios here:

climate-ref/packages/climate-ref-esmvaltool/src/climate_ref_esmvaltool/diagnostics/climate_at_global_warming_levels.py

Lines 72 to 76 in cf05fe9

	AddSupplementaryDataset(
	supplementary_facets={"experiment_id": "historical"},
	matching_facets=matching_facets,
	optional_matching_facets=tuple(),
	),

. Those variables will certainly be time-dependent.

[bouweandela]

On second thought, I wonder how bad the extra areacella files for the global warming levels diagnostic are. Some will get picked up by esmvaltool and others may not, but in the end it shouldn't affect the result.

[bouweandela]

I wonder why this execution disappeared

[bouweandela]

I tried to understand what is happening here, but this is getting rather complicated. Will have another look tomorrow.