Add MET pointstat configuration for the Met Office by JorgeBornemann · Pull Request #1079 · MetOffice/CSET

JorgeBornemann · 2025-01-27T09:10:17Z

Fixes #653

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

Documentation has been updated to reflect change.
New code has tests, and affected old tests have been updated.
All tests and CI checks pass.
Ensured the pull request title is descriptive.
Conda lock files have been updated if dependencies have changed.
Attributed any Generative AI, such as GitHub Copilot, used in this PR.
Marked the PR as ready to review.

…at-met-office-configuration

github-actions · 2025-01-27T09:11:42Z

JorgeBornemann · 2025-01-27T09:12:33Z

Initial implementation of NG-VER METPlus PointStat configurations for the Met Office.

jfrost-mo · 2025-01-27T09:20:24Z

Thanks Jorge, its great to see this!

I'll probably have a chance to review towards the end of the week, so let me know if there is anything to pay particular attention to.

JorgeBornemann · 2025-01-27T09:26:32Z

At the moment is just sanity check of the workflow structure and the apps/wrappers. If you think it is sound, it will need someone from the Met Office who has access to the observations and forecasts files to take it over, finish the coding and test thoroughly. Any change beyond the Met Office specific files, I need to be aware of it, as it will impact other sites.

JorgeBornemann · 2025-01-27T09:37:25Z

One thing to pay special attention to is whether there is in this branch information that should go in the restricted repo.

jfrost-mo

Per our discussion today, I've only reviewed the non .conf files, as the .conf files were copied from elsewhere.

Similarly, we should check with Rachel whether there is a more up-to-date branch that we should be using instead.

But otherwise this change looks sensible, and aside from a few things noted in the comments below, it should be fine to merge eventually.

cset-workflow/flow.cylc

cset-workflow/includes/metplus_grid_stat.cylc

jfrost-mo · 2026-01-26T11:13:24Z

cset-workflow/includes/metplus_grid_stat.cylc

+    {% elif CSET_CYCLING_MODE == "trial" %}
+        # Analyse from each forecast.
+        {{CSET_TRIAL_CYCLE_PERIOD}} = """
+        metplus_grid_stat => housekeeping_full


Suggested change

metplus_grid_stat => housekeeping_full

metplus_grid_stat => housekeeping

Same as above

cset-workflow/includes/metplus_point_stat.cylc

jfrost-mo · 2026-01-26T11:20:36Z

cset-workflow/includes/metplus_grid_stat.cylc

These tasks should all depend on setup_complete[^], which ensures all the cold_start stuff is complete (environment creation, etc.)

They should probably also depend on fetch_complete which is a per-cycle marker indicating that all the needed data has been retrieved.

Noted. We are still developing the METPlus side of the workflow without connection to the rest of the workflow, using our own canned data to minimise disruptions, but we will soon plug it into the main workflow and we will add the dependencies then.

cset-workflow/includes/metplus_point_stat.cylc

jfrost-mo · 2026-01-26T11:22:50Z

cset-workflow/app/metplus_point_stat/file/metoffice/pointstat_wrapper.sh

+
+#no set -xeu as check errors in next task
+
+#no set -xeu as check errors in next task


Suggested change

#no set -xeu as check errors in next task

Remove duplicate line.

This was a direct copy from the NG-VER config files. The current implementation does not use the wrapper script, it will instead use parametrized tasks.

jfrost-mo · 2026-01-26T11:26:06Z

cset-workflow/app/metplus_point_stat/file/metoffice/pointstat_wrapper.sh

+#no set -xeu as check errors in next task
+
+#no set -xeu as check errors in next task
+export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/data/users/cfrd/MET_requirements/MET9/lib/:/usr/lib64/:/net/project/ukmo/scitools/opt_scitools/conda/deployments/default-current/lib


While this is the Met Office specific file, could we abstract these paths out into the configuration? Especially as these paths are out of date.

No longer relevant with the new implementation.

jfrost-mo · 2026-01-26T11:29:27Z

cset-workflow/app/metplus_point_stat/file/metoffice/pointstat_wrapper.sh

CYLC_SUITE_DEF_PATH is a cylc 7ism. It should be replaced with CYLC_WORKFLOW_RUN_DIR.

Yep. Our reference NG-VER workflow is still cylc 7, but we won't be using the wrapper anyway.

jfrost-mo · 2026-02-18T08:46:52Z

@JorgeBornemann with #1906 being merged does this PR need updating and merging? Or are we still waiting on updated MET configurations?

JorgeBornemann · 2026-02-18T09:24:19Z

@JorgeBornemann with #1906 being merged does this PR need updating and merging? Or are we still waiting on updated MET configurations?

@jfrost-mo , I have to check with @RachelNorth. We have a sprint of sorts planned for the 23rd and 24th of February. We may be able to get the PR before then, but it will be done first thing on the 23rd at the latest.

jfrost-mo · 2026-02-18T10:03:39Z

OFFICIAL

but it will be done first thing on the 23rd at the latest.

Sounds good. We’re planning on cutting a release on the UK afternoon of the 23rd, so if it goes in before then it will be included. Otherwise it will be available via the development version and in future releases. OFFICIAL

JorgeBornemann · 2026-02-18T10:12:43Z

We’re planning on cutting a release on the UK afternoon of the 23rd, so if it goes in before then it will be included.

No big deal if it doesn't go in the release, as it will not have the full functionality.

JorgeBornemann · 2026-02-25T08:22:44Z

This functionality has already been implemented in #1918 and #1930. But some of the comments are still relevant. I will address them one by one, move the ones still relevant to a separate issue and close this one.

JorgeBornemann · 2026-02-26T01:41:35Z

These (an others) are now recorded on #1932, so I will close this pull request now as duplicated. @jfrost-mo do you want to be assigned to #1932 so you get notifications as we progress on it?

JorgeBornemann · 2026-02-26T01:47:09Z

Branch https://github.com/MetOffice/CSET/tree/653-add-pointstat-met-office-configuration deleted as is no longer relevant.

JorgeBornemann added 11 commits July 8, 2024 22:39

initial copy of apps

b5c3c3d

clean up

5084295

Merge branch 'main' of github.com:MetOffice/CSET into 653-add-pointst…

006f38d

…at-met-office-configuration

start alignment of configurations where possible

700c4c9

catch up with trunk

45d79ef

changes to includes for METPlus tasks

496d8f3

enable cycling over obstypes

1bf30ec

correct LEAD_SEQ value in ascii2nc

33540c8

add metoffice pointstat configuration files

f04ddbd

enable iterating over point-stat statistics

fa8404b

Merge branch 'main' of github.com:MetOffice/CSET into 653-add-pointst…

b29f746

…at-met-office-configuration

JorgeBornemann linked an issue Jan 27, 2025 that may be closed by this pull request

Add PointStat Met Office configuration #653

Closed

JorgeBornemann marked this pull request as draft January 27, 2025 09:10

JorgeBornemann requested a review from jfrost-mo January 27, 2025 09:12

jfrost-mo marked this pull request as ready for review January 27, 2025 09:17

jfrost-mo marked this pull request as draft January 27, 2025 09:19

jfrost-mo changed the title ~~653 add pointstat met office configuration~~ Add MET pointstat configuration for the Met Office Jul 24, 2025

jfrost-mo reviewed Jan 26, 2026

View reviewed changes

SGallagherMet added the R2O Trials Items of significance to the Met Office Research to Operations Team. label Feb 16, 2026

JorgeBornemann closed this Feb 26, 2026

JorgeBornemann deleted the 653-add-pointstat-met-office-configuration branch February 26, 2026 01:47

JorgeBornemann added the duplicate This issue or pull request already exists label Feb 26, 2026

	metplus_grid_stat => housekeeping_full
	metplus_grid_stat => housekeeping


		#no set -xeu as check errors in next task

		#no set -xeu as check errors in next task

Conversation

JorgeBornemann commented Jan 27, 2025 • edited by jfrost-mo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution checklist

Uh oh!

github-actions bot commented Jan 27, 2025

Uh oh!

JorgeBornemann commented Jan 27, 2025

Uh oh!

jfrost-mo commented Jan 27, 2025

Uh oh!

JorgeBornemann commented Jan 27, 2025

Uh oh!

JorgeBornemann commented Jan 27, 2025

Uh oh!

jfrost-mo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jfrost-mo commented Feb 18, 2026

Uh oh!

JorgeBornemann commented Feb 18, 2026

Uh oh!

jfrost-mo commented Feb 18, 2026 via email

Uh oh!

JorgeBornemann commented Feb 18, 2026

Uh oh!

JorgeBornemann commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JorgeBornemann commented Feb 26, 2026

Uh oh!

JorgeBornemann commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JorgeBornemann commented Jan 27, 2025 •

edited by jfrost-mo

Loading

JorgeBornemann commented Feb 25, 2026 •

edited

Loading