Merge stochastic physics (SPPT only) into GSL's develop branch by gsketefian · Pull Request #239 · ufs-community/MPAS-Model

gsketefian · 2026-03-31T18:58:51Z

This PR enables use of the SPPT scheme of stochastic physics into MPAS-Model. This is addressed in Issue #204.

Main changes:

Bring in the stochastic_physics code as a submodule.
Make changes to Makefiles, registry xml files, and fortran files to allow the SPPT stochastic scheme to bue used with MPAS.
Adds the stochastic_physics code as a submodule. There is a companion PR into the authoritative stochastic_physics repo here.

Note that this PR still requires make to be used to build MPAS-Model with stochastic physics. The ability to build with cmake will come in a separate PR.

Mandatory Questions

Does this PR include any additions or changes to external inputs (e.g., microphysics lookup tables, static data for gravity-wave drag -- things like that)?
- No
Does this PR require updating one or more baselines for the CI tests? If so, what?
- Yes. We would like to add at least one test that runs MPAS with SPPT enabled. We are also curious about how to include regression tests if those are used in this repo.

Priority Reviewers

Please list the developers/collaborators you'd like to prioritize for review

…he stochastic_physics submodule itself will be added in a separate commit.

…'t updated during latest merge of gsl/develop.

…he gsl/MPAS_stoch_physics in the stochastic_physics repo.

…teger seed workaround.

… smoke) from "output" to "output_smoke" (and file from "history..." to "history_smoke...". This is necessary because the stream name "output" is already taken by the main output stream, and having the same name for two different output streams apparently causes SMIOL I/O errors during the forecast (and incorrect/corrupted history*.nc files).

…) no stale stream_list.atmosphere files exist in the two default_inputs directories (one under core_atmosphere and the other immediately under the MPAS-Model top-level directory), and (2) for the atmosphere core, all files (stream-related as well as namelist) in the top-level directory are backed up before new such files are copied from the default_inputs directory back up a level into the top-level directory. Previously, existing (and thus possibly outdated) stream_list.atmosphere.* files in the top-level directory were not being replaced by newer ones in default_inputs, and that was causing unexpected (and wrong) behavior. Probably a similar fix is needed for the init_atmsophere core and maybe even other ones.

…ll as its output file) so it doesn't conflict with the default output stream.

…tochastic_physics's master branh will gradually happen.

…/MPAS_stoch_physics_try_merge_stoch_master

…tochastic_physics code know that the dycore being used is MPAS.

…ill instead be defined in the Makefile for stochastic_physics only (in a separate commit into the stochastic_physics repo) since it is only needed by the stochastic_physics submodule.

…e that is now added in the Makefile in stochastic_physics).

…h_master

…s the TEMPO tables.

…the top level of MPAS-Model.

gsketefian · 2026-03-31T19:00:40Z

src/core_atmosphere/physics/Makefile

@@ -65,7 +65,8 @@ core_physics_mmm: core_physics_init
 	(cd physics_mmm; $(MAKE) -f Makefile.mpas all)

 core_microphysics: core_physics_init core_physics_mmm


Changes in first line are to copy ccn_activate.bin into the top-level MPAS-Model directory, just as other microphysics files are copied there during a make invocation (e.g. MP_THOMPSON_*).

gsketefian · 2026-03-31T19:02:46Z

src/core_atmosphere/physics/Makefile


 core_microphysics: core_physics_init core_physics_mmm
-	(cd physics_noaa/TEMPO; $(MAKE) -f Makefile.mpas)
+	(cd physics_noaa/TEMPO; $(MAKE) -f Makefile.mpas; cp tables/ccn_activate.bin ../../../../../)


The following new line builds TEMPO's run_build_tables executable, which is needed to generate the tables for TEMPO. This is similar to the building of the build_tables executable for THOMPSON microphysics.

Not sure why this wasn't originally included. Also, not sure if this work with all machines and compilers, but it works on Hera.

@clark-evans Not sure if you're the right person to ask this. If not, please let me know who to ask (Anders?).

I'm wondering why the original version of this Makefile does not build the executable for generating the TEMPO MP tables. I tried to include it via this line, but it is failing during CI testing because the sub-makefile called (Makefile.intel) is apparently only for intel, but one of the CI tests is with gnu FORTRAN.

Without this executable (named run_build_tables) a user will not be able to generate the tables needed to run MPAS with TEMPO. Note that an executable for generating tables for THOMPSON MP (named build_tables) is generated during the make, so it would be nice to have one for TEMPO as well.

I will comment out this line for now so the CI tests can proceed, but it would be nice to understand why building run_build_tables (regardless of compiler) is not part of the original Makefile.

Hi @gsketefian -- I think this is handled in src/core_atmosphere/utils/Makefile, where directives for compiling build_tables_tempo are provided alongside those for build_tables. This is called when core_atmosphere is built.

Tagging @AndersJensen-NOAA to correct me in case I'm wrong, though...

@clark-evans @AndersJensen-NOAA As far as I can tell, that Makefile builds the executable for Thompson MP but not the one for TEMPO, probably intentionally. In my build log, I can see the fortran files for Thompson (atmphys_build_tables_thompson.F and build_tables.F) being compiled but not the ones for TEMPO (atmphys_build_tables_tempo.F and build_tables_tempo.F).

@clark-evans @AndersJensen-NOAA I talked to Anders today about this, and he pointed out that there is also a Makefile for gfortran (Makefile.gfortran) for building the TEMPO executable to generate MP tables. So I added the variable MAKEFILE_SUFFIX to distinguish between intel and gfortran.

Btw, the code in src/core_atmosphere/utils/ related to TEMPO is outdated since the subroutine interfaces for the latest TEMPO are different than the ones used in the .F files in that directory. So another option for building the executable for TEMPO table generation is to update this directory (specifically atmphys_build_tables_tempo.F, build_tables_tempo.F, and Makefile) to match the latest TEMPO, but that is a job best suited for @AndersJensen-NOAA (and not necessary for this PR).

gsketefian · 2026-03-31T19:03:47Z

src/core_atmosphere/physics/Makefile

@@ -290,6 +291,7 @@ clean:
 	( cd physics_noahmp/src; $(MAKE) clean )
 	( cd physics_noahmp/utility; $(MAKE) clean )
 	( if [ -d physics_noaa/TEMPO ]; then cd physics_noaa/TEMPO; $(MAKE) -f Makefile.mpas clean; fi )


The following is to clean the run_build_tables executable and related files generated by the new line above in this Makefile.

gsketefian · 2026-03-31T19:06:37Z

src/core_atmosphere/physics/registry.chemistry.xml

@@ -199,9 +199,9 @@
 <!-- **************************************************************************************** -->

   <streams>


This was necessary to avoid SMIOL errors apparently due to another stream named output defined in the main Registry.xml. This error causes the output .nc files to contain only the smoke variables defined here and none of the output variables for the output stream in the main Registry.xml. The output file is also renamed so two different streams are not writing to the same file.

gsketefian · 2026-03-31T19:07:30Z

src/core_atmosphere/Makefile

@@ -28,6 +28,7 @@ core_reg:

 core_input_gen:
 	if [ ! -e default_inputs ]; then  mkdir default_inputs; fi


This is to remove any stale stream files (streams.atmosphere and stream_list.atmosphere.*) in the default_inputs directory under core_atmosphere generated by a previous invocation of make.

gsketefian · 2026-03-31T19:09:34Z

src/core_atmosphere/Makefile

@@ -37,12 +38,21 @@ gen_includes: core_reg

 post_build:
 	if [ ! -e $(ROOT_DIR)/default_inputs ]; then mkdir $(ROOT_DIR)/default_inputs; fi


This is to remove any stale stream files (streams.atmosphere and stream_list.atmosphere.*) in the default_inputs directory under the top-level MPAS-Model directory generated by a previous invocation of make.

guoqing-noaa · 2026-03-31T19:10:43Z

src/core_atmosphere/physics/registry.chemistry.xml

+      <stream name="output_smoke"
              type="output"
-              filename_template="history.$Y-$M-$D_$h.$m.$s.nc"
+              filename_template="history_smoke.$Y-$M-$D_$h.$m.$s.nc"


@gsketefian This change will break the current rrfs-workflow.v2 (with smoke and dust) as discussed in #196 (comment)

We need to discuss this and have an agreement before making this change. Thanks!

gsketefian · 2026-03-31T19:14:12Z

src/core_atmosphere/Makefile

 post_build:
 	if [ ! -e $(ROOT_DIR)/default_inputs ]; then mkdir $(ROOT_DIR)/default_inputs; fi
+	rm -f $(ROOT_DIR)/default_inputs/streams.$(CORE) $(ROOT_DIR)/default_inputs/stream_list.$(CORE).*
 	cp default_inputs/* $(ROOT_DIR)/default_inputs/.


Previously, files (e.g. stream-related files such as streams.atmosphere and stream_list.atmosphere.*) were not copied from the default_inputs directory under the top-level directory (MPAS-Model) if files with the same name already exist under MPAS-Model. This just seems wrong since we want to update those files with each make invocation. The new code makes a backup of the existing file (e.g. <file_name>.old01, <file_name>.old02, etc) before replacing the existing file with the new one from default_inputs.

gsketefian · 2026-03-31T19:15:25Z

src/Makefile

@@ -9,7 +9,7 @@ else
 all: mpas

 mpas: $(AUTOCLEAN_DEPS) externals frame ops dycore drver


Note that the new line here now uses esmf_time_f91 instead of esmf_time_f90.

@NingWang325 @clark-evans These intel mkl libraries are generating the following errors in the CI build tests:

With gnu compiler:

/usr/bin/ld: cannot find -lmkl_intel_lp64: No such file or directory /usr/bin/ld: cannot find -lmkl_core: No such file or directory /usr/bin/ld: cannot find -lmkl_sequential: No such file or directory

With intel compiler:

ld: cannot find -lmkl_intel_lp64: No such file or directory ld: cannot find -lmkl_core: No such file or directory ld: cannot find -lmkl_sequential: No such file or directory

First, I thought this was due to gnu not recognizing these intel libraries, but it is also happening with the intel compiler. Do we need to change the modules when running the CI tests?

Thanks.

…hat generates the data tables for the TEMPO microphysics scheme. This is necessary because the makefile invoked (Makefile.intel) is only for the intel fortran compiler but the CI tests for the repo include build tests with gnu fortran (which are failing with these lines present).

…building TEMPO's executable for generating microphysics tables.

… Math Kernel Library (MKL).

gsketefian and others added 30 commits September 12, 2025 12:36

Add changes to MPAS-Model code necessary to run stochastic physics. T…

6f687ea

…he stochastic_physics submodule itself will be added in a separate commit.

Add stochastic physics as a submodule.

bb262cd

Bug fix: Add missing lines for stochastic physics to Registry.xml.

e5bde4d

Change hash of stochastic_physics.

293e081

Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics

abc034b

Update hashes of GFL and MYNN-EDMF submodules. Not sure why this wasn…

cee787e

…'t updated during latest merge of gsl/develop.

A new version of SPPT merged into the latest gsl/MPAS_stoch_physics.

c93caca

Update hash of stochastic_physics submodule to point to the head of t…

845ff87

…he gsl/MPAS_stoch_physics in the stochastic_physics repo.

Update hash of stochastic_physics to the hash from Ning for 8-byte in…

995a0cf

…teger seed workaround.

Change the name of the output stream in registry.chemistry.xml (as we…

cea828b

…ll as its output file) so it doesn't conflict with the default output stream.

Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics

9d0eb11

Switch hash of stochastic_physics to branch in which the merge with s…

8428af8

…tochastic_physics's master branh will gradually happen.

Merge remote-tracking branch 'origin/gsl/MPAS_stoch_physics' into gsl…

a0700bf

…/MPAS_stoch_physics_try_merge_stoch_master

Fix hash of stochastic_physics.

f8fea43

In the top-level Makefile, add a flag to CPPFLAGS that will let the s…

4060353

…tochastic_physics code know that the dycore being used is MPAS.

Update hash of stochastic_physics.

9bdd0de

Update hash of stochastic_physics.

57f2ccf

Remove definition of macro MPAS_STOCH from main MPAS Makefile. This w…

6f5de0a

…ill instead be defined in the Makefile for stochastic_physics only (in a separate commit into the stochastic_physics repo) since it is only needed by the stochastic_physics submodule.

Update hash of stochastic_physics.

662e9c2

Remove commented-out lines in Makefile that refer to MPAS_STOCH (sinc…

b4c22a1

…e that is now added in the Makefile in stochastic_physics).

Update hash of stochastic_physics.

d2708cf

Update hash of stochastic_physics.

ec08395

Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics_try_merge_stoc…

ae7c0d7

…h_master

Add lines to physics Makefile to also build the executable that build…

a11e46f

…s the TEMPO tables.

Remove commented-out lines in Makefiles.

d5a2107

Clean up comments.

365959c

Get rid of comments in Makefile for brevity.

795dd8c

Update hash of stochastic_physics.

722cac1

Modify Makefile so that TEMPO's ccn_activate.bin file gets copied to …

aaf05d0

…the top level of MPAS-Model.

gsketefian commented Mar 31, 2026

View reviewed changes

guoqing-noaa reviewed Mar 31, 2026

View reviewed changes

gsketefian commented Mar 31, 2026

View reviewed changes

gsketefian mentioned this pull request Mar 31, 2026

Enable SPPT with the MPAS dycore NOAA-PSL/stochastic_physics#97

Draft

gsketefian added 13 commits March 31, 2026 15:01

Update hash of stochastic_physics.

b3a8e0c

Add code to choose between Makefile.gfortran and Makefile.intel when …

e3dedb8

…building TEMPO's executable for generating microphysics tables.

Bug fixes and improvements to Makefiles.

91bcfba

Update hash of stochastic_physics.

7782ece

Add lines to GitHub Actions workflow file that will link in the Intel…

0c2d253

… Math Kernel Library (MKL).

Find location of MKL library on GitHub runner.

2934e47

Second attempt to find MKL library.

97a98ec

Install MKL library in Github Actions.

fe3720c

Fix MKL locations in Github Actions runner.

0f4d720

Bug fixes to Github Actions workflow file.

1b05207

More debugging of GitHub Actions workflow file.

18f58f3

More debugging.

82c99cd

		@@ -65,7 +65,8 @@ core_physics_mmm: core_physics_init
		(cd physics_mmm; $(MAKE) -f Makefile.mpas all)

		core_microphysics: core_physics_init core_physics_mmm

		@@ -199,9 +199,9 @@
		<!-- **************************************************************************************** -->

		<streams>

		@@ -28,6 +28,7 @@ core_reg:

		core_input_gen:
		if [ ! -e default_inputs ]; then mkdir default_inputs; fi

		@@ -37,12 +38,21 @@ gen_includes: core_reg

		post_build:
		if [ ! -e $(ROOT_DIR)/default_inputs ]; then mkdir $(ROOT_DIR)/default_inputs; fi

		@@ -9,7 +9,7 @@ else
		all: mpas

		mpas: $(AUTOCLEAN_DEPS) externals frame ops dycore drver

Conversation

gsketefian commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Mandatory Questions

Priority Reviewers

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsketefian Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsketefian Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsketefian Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsketefian Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gsketefian commented Mar 31, 2026 •

edited

Loading

gsketefian Mar 31, 2026 •

edited

Loading

gsketefian Apr 1, 2026 •

edited

Loading

gsketefian Mar 31, 2026 •

edited

Loading

gsketefian Mar 31, 2026 •

edited

Loading