Skip to content

Merge stochastic physics (SPPT only) into GSL's develop branch #239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gsketefian wants to merge 44 commits into
base: gsl/develop
Choose a base branch
from
+127 −10
Draft

Merge stochastic physics (SPPT only) into GSL's develop branch #239

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
6f687ea
Add changes to MPAS-Model code necessary to run stochastic physics. …
gsketefian Sep 12, 2025
bb262cd
Add stochastic physics as a submodule.
gsketefian Sep 12, 2025
e5bde4d
Bug fix: Add missing lines for stochastic physics to Registry.xml.
gsketefian Sep 16, 2025
293e081
Change hash of stochastic_physics.
gsketefian Sep 16, 2025
abc034b
Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics
gsketefian Nov 13, 2025
cee787e
Update hashes of GFL and MYNN-EDMF submodules. Not sure why this was…
gsketefian Nov 15, 2025
c93caca
A new version of SPPT merged into the latest gsl/MPAS_stoch_physics.
NingWang325 Nov 24, 2025
845ff87
Update hash of stochastic_physics submodule to point to the head of t…
gsketefian Dec 19, 2025
995a0cf
Update hash of stochastic_physics to the hash from Ning for 8-byte in…
gsketefian Feb 2, 2026
5cf356a
Change name of and file for output stream for chemistry (and dust and…
gsketefian Feb 4, 2026
d63788d
Modifications to Makefile to make sure that after make is invoked, (1…
gsketefian Feb 25, 2026
cea828b
Change the name of the output stream in registry.chemistry.xml (as we…
gsketefian Feb 25, 2026
9d0eb11
Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics
gsketefian Feb 25, 2026
8428af8
Switch hash of stochastic_physics to branch in which the merge with s…
gsketefian Mar 11, 2026
a0700bf
Merge remote-tracking branch 'origin/gsl/MPAS_stoch_physics' into gsl…
gsketefian Mar 11, 2026
f8fea43
Fix hash of stochastic_physics.
gsketefian Mar 11, 2026
4060353
In the top-level Makefile, add a flag to CPPFLAGS that will let the s…
gsketefian Mar 14, 2026
9bdd0de
Update hash of stochastic_physics.
gsketefian Mar 14, 2026
57f2ccf
Update hash of stochastic_physics.
gsketefian Mar 14, 2026
6f5de0a
Remove definition of macro MPAS_STOCH from main MPAS Makefile. This …
gsketefian Mar 24, 2026
662e9c2
Update hash of stochastic_physics.
gsketefian Mar 24, 2026
b4c22a1
Remove commented-out lines in Makefile that refer to MPAS_STOCH (sinc…
gsketefian Mar 24, 2026
d2708cf
Update hash of stochastic_physics.
gsketefian Mar 24, 2026
ec08395
Update hash of stochastic_physics.
gsketefian Mar 27, 2026
ae7c0d7
Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics_try_merge_stoc…
gsketefian Mar 27, 2026
a11e46f
Add lines to physics Makefile to also build the executable that build…
gsketefian Mar 29, 2026
d5a2107
Remove commented-out lines in Makefiles.
gsketefian Mar 30, 2026
365959c
Clean up comments.
gsketefian Mar 30, 2026
795dd8c
Get rid of comments in Makefile for brevity.
gsketefian Mar 30, 2026
722cac1
Update hash of stochastic_physics.
gsketefian Mar 30, 2026
aaf05d0
Modify Makefile so that TEMPO's ccn_activate.bin file gets copied to …
gsketefian Mar 30, 2026
b3a8e0c
Update hash of stochastic_physics.
gsketefian Mar 31, 2026
462953d
Comment out lines that try to build the run_build_tables executable t…
gsketefian Mar 31, 2026
e3dedb8
Add code to choose between Makefile.gfortran and Makefile.intel when …
gsketefian Apr 1, 2026
91bcfba
Bug fixes and improvements to Makefiles.
gsketefian Apr 2, 2026
7782ece
Update hash of stochastic_physics.
gsketefian Apr 2, 2026
0c2d253
Add lines to GitHub Actions workflow file that will link in the Intel…
gsketefian Apr 2, 2026
2934e47
Find location of MKL library on GitHub runner.
gsketefian Apr 2, 2026
97a98ec
Second attempt to find MKL library.
gsketefian Apr 2, 2026
fe3720c
Install MKL library in Github Actions.
gsketefian Apr 2, 2026
0f4d720
Fix MKL locations in Github Actions runner.
gsketefian Apr 2, 2026
1b05207
Bug fixes to Github Actions workflow file.
gsketefian Apr 2, 2026
18f58f3
More debugging of GitHub Actions workflow file.
gsketefian Apr 3, 2026
82c99cd
More debugging.
gsketefian Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion .github/workflows/build_mpas_intel.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,29 @@ jobs:
install-oneapi: false
mpi-wrapper-setup: classic

- name: Install MKL
run: |
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \
| gpg --dearmor \
| sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" \
| sudo tee /etc/apt/sources.list.d/oneAPI.list
sudo apt-get update -q
sudo apt-get install -y intel-oneapi-mkl-devel

- name: Find MKL library location
run: |
echo "=== /opt/intel/oneapi contents ==="
ls /opt/intel/oneapi/ 2>/dev/null || echo "No /opt/intel/oneapi dir"
echo "=== broad libmkl search ==="
find /opt/intel /usr -name "libmkl_core*" 2>/dev/null | head -20 || echo "libmkl_core not found"
echo "=== installed intel-oneapi packages ==="
dpkg -l | grep intel-oneapi || echo "No intel-oneapi packages found"
echo "=== MKLROOT / MKL_ROOT env ==="
echo "MKLROOT=${MKLROOT}"
echo "MKL_ROOT=${MKL_ROOT}"
echo "Bye"

- name: Cache Intel
id: cache-INTEL
uses: actions/cache@v4
Expand Down Expand Up @@ -129,6 +152,10 @@ jobs:
echo "LD_LIBRARY_PATH=${AUTOTOOLS}/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
echo "PATH=${PNETCDF}/bin:$PATH" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=${PNETCDF}/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
echo "MKLROOT=/opt/intel/oneapi/mkl/2025.3" >> $GITHUB_ENV
echo "MKL_LIBDIR=/opt/intel/oneapi/mkl/2025.3/lib" >> $GITHUB_ENV
echo "LIBRARY_PATH=/opt/intel/oneapi/mkl/2025.3/lib:$LIBRARY_PATH" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=/opt/intel/oneapi/mkl/2025.3/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV

- name: Build MPAS Standalone (make)
if: matrix.build-system == 'make'
Expand All @@ -144,4 +171,4 @@ jobs:
mkdir build
cd build
cmake -DMPAS_DOUBLE_PRECISION=OFF -DMPAS_CORES="init_atmosphere;atmosphere" ..
make -j8
make -j8
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,6 @@
[submodule "src/core_atmosphere/physics/physics_mmm"]
path = src/core_atmosphere/physics/physics_mmm
url = https://github.com/NCAR/MMM-physics.git
[submodule "src/core_atmosphere/stochastic_physics"]
path = src/core_atmosphere/stochastic_physics
url = https://github.com/dtcenter/stochastic_physics.git
2 changes: 1 addition & 1 deletion src/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ else
all: mpas

mpas: $(AUTOCLEAN_DEPS) externals frame ops dycore drver
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the new line here now uses esmf_time_f91 instead of esmf_time_f90.

Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NingWang325 @clark-evans These intel mkl libraries are generating the following errors in the CI build tests:

With gnu compiler:

/usr/bin/ld: cannot find -lmkl_intel_lp64: No such file or directory
/usr/bin/ld: cannot find -lmkl_core: No such file or directory
/usr/bin/ld: cannot find -lmkl_sequential: No such file or directory

With intel compiler:

ld: cannot find -lmkl_intel_lp64: No such file or directory
ld: cannot find -lmkl_core: No such file or directory
ld: cannot find -lmkl_sequential: No such file or directory

First, I thought this was due to gnu not recognizing these intel libraries, but it is also happening with the intel compiler. Do we need to change the modules when running the CI tests?

Thanks.

$(LINKER) $(LDFLAGS) -o $(EXE_NAME) driver/*.o -L. -ldycore -lops -lframework $(LIBS) -I./external/esmf_time_f90 -L./external/esmf_time_f90 -lesmf_time
$(LINKER) $(LDFLAGS) -o $(EXE_NAME) driver/*.o -L. -ldycore -lops -lframework $(LIBS) -I./external/esmf_time_f91 -L./external/esmf_time_f90 -lesmf_time -lmkl_intel_lp64 -lmkl_core -lmkl_sequential

externals: $(AUTOCLEAN_DEPS)
( cd external; $(MAKE) FC="$(FC)" SFC="$(SFC)" CC="$(CC)" SCC="$(SCC)" FFLAGS="$(FFLAGS)" CFLAGS="$(CFLAGS)" CPP="$(CPP)" NETCDF="$(NETCDF)" CORE="$(CORE)" all )
Expand Down
24 changes: 20 additions & 4 deletions src/core_atmosphere/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ core_reg:

core_input_gen:
if [ ! -e default_inputs ]; then mkdir default_inputs; fi
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to remove any stale stream files (streams.atmosphere and stream_list.atmosphere.*) in the default_inputs directory under core_atmosphere generated by a previous invocation of make.

rm -f default_inputs/streams.atmosphere default_inputs/stream_list.atmosphere.*
( cd default_inputs; $(NL_GEN) ../Registry_processed.xml namelist.atmosphere in_defaults=true )
( cd default_inputs; $(ST_GEN) ../Registry_processed.xml streams.atmosphere stream_list.atmosphere. listed in_defaults=true)

Expand All @@ -37,12 +38,21 @@ gen_includes: core_reg

post_build:
if [ ! -e $(ROOT_DIR)/default_inputs ]; then mkdir $(ROOT_DIR)/default_inputs; fi
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to remove any stale stream files (streams.atmosphere and stream_list.atmosphere.*) in the default_inputs directory under the top-level MPAS-Model directory generated by a previous invocation of make.

rm -f $(ROOT_DIR)/default_inputs/streams.$(CORE) $(ROOT_DIR)/default_inputs/stream_list.$(CORE).*
cp default_inputs/* $(ROOT_DIR)/default_inputs/.
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, files (e.g. stream-related files such as streams.atmosphere and stream_list.atmosphere.*) were not copied from the default_inputs directory under the top-level directory (MPAS-Model) if files with the same name already exist under MPAS-Model. This just seems wrong since we want to update those files with each make invocation. The new code makes a backup of the existing file (e.g. <file_name>.old01, <file_name>.old02, etc) before replacing the existing file with the new one from default_inputs.

( cd $(ROOT_DIR)/default_inputs; for FILE in `ls -1`; do if [ ! -e ../$$FILE ]; then cp $$FILE ../.; fi; done )
( cd $(ROOT_DIR)/default_inputs; for FILE in `ls -1`; do \
if [ -e ../$$FILE ]; then \
N=1; \
while [ -e "../$$FILE.old$$(printf '%03d' $$N)" ]; do N=$$((N+1)); done; \
mv "../$$FILE" "../$$FILE.old$$(printf '%03d' $$N)"; \
fi; \
cp $$FILE ../.; \
done )

physcore: mpas_atm_dimensions.o
( cd physics; $(MAKE) all )
( mkdir libphys; cd libphys; ar -x ../physics/libphys.a )
( cd stochastic_physics; $(MAKE) -f Makefile all )
( mkdir libphys; cd libphys; ar -x ../physics/libphys.a; ar -x ../stochastic_physics/libstochphys.a )
( cd ../..; ln -sf ./src/core_atmosphere/physics/physics_wrf/files/*TBL .)
( cd ../..; ln -sf ./src/core_atmosphere/physics/physics_wrf/files/*DATA* .)
( cd ../..; ln -sf ./src/core_atmosphere/physics/physics_noahmp/parameters/*TBL .)
Expand All @@ -68,6 +78,12 @@ mpas_atm_dimensions.o:
clean:
( cd physics; $(MAKE) clean )
( cd dynamics; $(MAKE) clean )
@# The stochastic_physics directory contains both a makefile (lower case
@# "m") and a Makefile (upper case "M"). The former is used for FV3 and
@# the latter for MPAS. The default for "make" is to use "makefile", but
@# here, we want to use the one for MPAS (with an upper case "M"). Indicate
@# that using the -f flag.
( cd stochastic_physics; $(MAKE) -f Makefile clean )
( cd diagnostics; $(MAKE) clean )
( cd utils; $(MAKE) clean )
( cd ../..; rm -f *TBL )
Expand All @@ -84,7 +100,7 @@ clean:
$(RM) $@ $*.mod
ifeq "$(GEN_F90)" "true"
$(CPP) $(CPPFLAGS) $(PHYSICS) $(CPPINCLUDES) -I./inc $< > $*.f90
$(FC) $(FFLAGS) -c $*.f90 $(FCINCLUDES) -I../framework -I../operators -I./physics -I./dynamics -I./diagnostics -I./physics/physics_wrf -I./physics/physics_mmm -I./physics/physics_noaa/UGWP -I../external/esmf_time_f90
$(FC) $(FFLAGS) -c $*.f90 $(FCINCLUDES) -I../framework -I../operators -I./physics -I./dynamics -I./diagnostics -I./physics/physics_wrf -I./physics/physics_mmm -I./physics/physics_noaa/UGWP -I./stochastic_physics -I../external/esmf_time_f90
else
$(FC) $(CPPFLAGS) $(PHYSICS) $(FFLAGS) -c $*.F $(CPPINCLUDES) $(FCINCLUDES) -I./inc -I../framework -I../operators -I./physics -I./dynamics -I./diagnostics -I./physics/physics_wrf -I./physics/physics_mmm -I./physics/physics_noaa/UGWP -I../external/esmf_time_f90
$(FC) $(CPPFLAGS) $(PHYSICS) $(FFLAGS) -c $*.F $(CPPINCLUDES) $(FCINCLUDES) -I./inc -I../framework -I../operators -I./physics -I./dynamics -I./diagnostics -I./physics/physics_wrf -I./physics/physics_mmm -I./physics/physics_noaa/UGWP -I./stochastic_physics -I../external/esmf_time_f90
endif
12 changes: 12 additions & 0 deletions src/core_atmosphere/Registry.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@
description="The number of atmospheric layers"/>
<dim name="nVertLevelsP1" definition="nVertLevels+1"
description="The number of atmospheric levels, always one more than the number of layers"/>
<dim name="lon_for" definition="504"
description="The number of longitude points for the Gaussian grid"/>
<dim name="lat_leg" definition="248"
description="The number of latitude points for the Gaussian grid"/>

#ifdef DO_PHYSICS
<dim name="nMonths" definition="namelist:months"
Expand Down Expand Up @@ -979,6 +983,8 @@
<var name="ke"/>
<var name="uReconstructZonal"/>
<var name="uReconstructMeridional"/>
<var name="stoch_pattern_sppt"/>
<var name="stoch_pattern_gg"/>

<!-- Begin Ertel PV diagnostics defined in diagnostics/Registry_pv.xml -->
<var name="ertel_pv"/>
Expand Down Expand Up @@ -4162,6 +4168,11 @@
description="Total cell-centered meridional wind tendency from physics"
persistence="scratch" />

<var name="stoch_pattern_sppt" type="real" dimensions="nVertLevels nCells Time" units="%"
description="Perturbation pattern for stochastic physics SPPT scheme" />

<var name="stoch_pattern_gg" type="real" dimensions="lon_for lat_leg Time" units="%"
description="Perturbation pattern for stochastic physics, on Gaussian grid" />
#ifdef DO_PHYSICS
<!-- ================================================================================================== -->
<!-- TENDENCIES FROM PARAMETERIZATION OF CLOUD MICROPHYSICS: -->
Expand Down Expand Up @@ -4672,5 +4683,6 @@
#include "physics/Registry_noahmp.xml"
#include "physics/Registry_tempo.xml"
#include "physics/registry.chemistry.xml"
#include "stochastic_physics/Registry_stoch_physics.xml"
#endif
</registry>
2 changes: 1 addition & 1 deletion src/core_atmosphere/dynamics/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@ ifeq "$(GEN_F90)" "true"
$(CPP) $(CPPFLAGS) $(PHYSICS) $(CPPINCLUDES) $< > $*.f90
$(FC) $(FFLAGS) -c $*.f90 $(FCINCLUDES) -I.. -I../../framework -I../../operators -I../physics -I../physics/physics_wrf -I../physics/physics_mmm -I../../external/esmf_time_f90
else
$(FC) $(CPPFLAGS) $(PHYSICS) $(FFLAGS) -c $*.F $(CPPINCLUDES) $(FCINCLUDES) -I.. -I../../framework -I../../operators -I../physics -I../physics/physics_wrf -I../physics/physics_mmm -I../../external/esmf_time_f90
$(FC) $(CPPFLAGS) $(PHYSICS) $(FFLAGS) -c $*.F $(CPPINCLUDES) $(FCINCLUDES) -I.. -I../../framework -I../../operators -I../physics -I../physics/physics_wrf -I../physics/physics_mmm -I../stochastic_physics -I../../external/esmf_time_f90
endif
20 changes: 20 additions & 0 deletions src/core_atmosphere/dynamics/mpas_atm_time_integration.F
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ module atm_time_integration
use mpas_atmphys_driver_microphysics
use mpas_atmphys_todynamics
use mpas_atmphys_utilities
use mpas_stochastic_physics, only : stochastic_physics_pattern_apply, dosppt
#endif

use mpas_atm_boundaries, only : nSpecZone, nRelaxZone, nBdyZone, mpas_atm_get_bdy_state, mpas_atm_get_bdy_tend ! regional_MPAS addition
Expand Down Expand Up @@ -880,6 +881,8 @@ subroutine atm_srk3(domain, dt, itimestep, exchange_halo_group)
#endif

real (kind=RKIND) :: time_dyn_step
character(len=32) :: tend_names(4)
integer :: ierr
logical, parameter :: debug = .false.


Expand Down Expand Up @@ -1077,11 +1080,28 @@ subroutine atm_srk3(domain, dt, itimestep, exchange_halo_group)
call mpas_timer_start('physics_get_tend')
rk_step = 1
dynamics_substep = 1

! apply random perturbation pattern to the tendency
tend_names(1) = "rucuten"
tend_names(2) = "rvcuten"
tend_names(3) = "rublten"
tend_names(4) = "rvblten"
if (dosppt(domain)) then
call stochastic_physics_pattern_apply(domain, 4, tend_names, ierr)
endif

call physics_get_tend( block, mesh, state, diag, tend, tend_physics, &
block % configs, rk_step, dynamics_substep, &
tend_ru_physics, tend_rtheta_physics, tend_rho_physics, &
exchange_halo_group )
call mpas_timer_stop('physics_get_tend')

! apply random perturbation pattern to the tendency
tend_names(1) = "tend_rtheta_physics"
! tend_names(2) = "tend_rho_physics"
if (dosppt(domain)) then
call stochastic_physics_pattern_apply(domain, 1, tend_names, ierr)
endif
#else
#ifndef MPAS_CAM_DYCORE
!
Expand Down
10 changes: 10 additions & 0 deletions src/core_atmosphere/mpas_atm_core.F
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ function atm_core_init(domain, startTimeStamp) result(ierr)
use mpas_attlist, only : mpas_modify_att
use mpas_string_utils, only : mpas_string_replace
use mpas_atm_halos, only: atm_build_halo_groups, exchange_halo_group
use mpas_stochastic_physics, only : stochastic_physics_pattern_init, dosppt

implicit none

Expand Down Expand Up @@ -301,6 +302,11 @@ function atm_core_init(domain, startTimeStamp) result(ierr)
! Prepare the dynamics for integration
!
call mpas_atm_dynamics_init(domain)
!
! init stochastic pattern generation
if (dosppt(domain)) then
call stochastic_physics_pattern_init (domain, ierr)
endif

end function atm_core_init

Expand Down Expand Up @@ -1014,6 +1020,7 @@ subroutine atm_do_timestep(domain, dt, itimestep)
use mpas_atmphys_update
#endif
use mpas_atm_halos, only: exchange_halo_group
use mpas_stochastic_physics, only : stochastic_physics_pattern_adv, dosppt

implicit none

Expand Down Expand Up @@ -1046,6 +1053,9 @@ subroutine atm_do_timestep(domain, dt, itimestep)
endif
#endif

if (dosppt(domain)) then
call stochastic_physics_pattern_adv(domain, itimestep, ierr)
endif
call atm_timestep(domain, dt, currTime, itimestep, exchange_halo_group)

end subroutine atm_do_timestep
Expand Down
30 changes: 29 additions & 1 deletion src/core_atmosphere/physics/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,26 @@ OBJS_init = \
mpas_atmphys_functions.o \
mpas_atmphys_utilities.o

# Set TEMPO_TBLS_MAKEFILE_SUFFIX needed to build TEMPO's executable for
# generating microphysics tables. Note that when the "clean" target is
# invoked by upstream Makefies, FC_SERIAL and FC_PARALLEL will not be set,
# in which case TEMPO_TBLS_MAKEFILE_SUFFIX will be set to its default
# value of "none".
TEMPO_TBLS_MAKEFILE_SUFFIX = none

ifeq "$(FC_SERIAL)" "gfortran"
ifeq "$(FC_PARALLEL)" "mpif90"
TEMPO_TBLS_MAKEFILE_SUFFIX = gfortran
endif
endif

ifeq "$(FC_SERIAL)" "ifort"
ifeq "$(FC_PARALLEL)" "mpiifort"
TEMPO_TBLS_MAKEFILE_SUFFIX = intel
endif
endif
$(info TEMPO_TBLS_MAKEFILE_SUFFIX = $(TEMPO_TBLS_MAKEFILE_SUFFIX))

OBJS = \
mpas_atmphys_camrad_init.o \
mpas_atmphys_control.o \
Expand Down Expand Up @@ -65,7 +85,8 @@ core_physics_mmm: core_physics_init
(cd physics_mmm; $(MAKE) -f Makefile.mpas all)

core_microphysics: core_physics_init core_physics_mmm
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes in first line are to copy ccn_activate.bin into the top-level MPAS-Model directory, just as other microphysics files are copied there during a make invocation (e.g. MP_THOMPSON_*).

(cd physics_noaa/TEMPO; $(MAKE) -f Makefile.mpas)
(cd physics_noaa/TEMPO; $(MAKE) -f Makefile.mpas; cp tables/ccn_activate.bin ../../../../../)
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following new line builds TEMPO's run_build_tables executable, which is needed to generate the tables for TEMPO. This is similar to the building of the build_tables executable for THOMPSON microphysics.

Not sure why this wasn't originally included. Also, not sure if this work with all machines and compilers, but it works on Hera.

Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clark-evans Not sure if you're the right person to ask this. If not, please let me know who to ask (Anders?).

I'm wondering why the original version of this Makefile does not build the executable for generating the TEMPO MP tables. I tried to include it via this line, but it is failing during CI testing because the sub-makefile called (Makefile.intel) is apparently only for intel, but one of the CI tests is with gnu FORTRAN.

Without this executable (named run_build_tables) a user will not be able to generate the tables needed to run MPAS with TEMPO. Note that an executable for generating tables for THOMPSON MP (named build_tables) is generated during the make, so it would be nice to have one for TEMPO as well.

I will comment out this line for now so the CI tests can proceed, but it would be nice to understand why building run_build_tables (regardless of compiler) is not part of the original Makefile.

Copy link
Copy Markdown
Collaborator

@clark-evans clark-evans Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @gsketefian -- I think this is handled in src/core_atmosphere/utils/Makefile, where directives for compiling build_tables_tempo are provided alongside those for build_tables. This is called when core_atmosphere is built.

Tagging @AndersJensen-NOAA to correct me in case I'm wrong, though...

Copy link
Copy Markdown
Author

@gsketefian gsketefian Apr 1, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clark-evans @AndersJensen-NOAA As far as I can tell, that Makefile builds the executable for Thompson MP but not the one for TEMPO, probably intentionally. In my build log, I can see the fortran files for Thompson (atmphys_build_tables_thompson.F and build_tables.F) being compiled but not the ones for TEMPO (atmphys_build_tables_tempo.F and build_tables_tempo.F).

Copy link
Copy Markdown
Author

@gsketefian gsketefian Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clark-evans @AndersJensen-NOAA I talked to Anders today about this, and he pointed out that there is also a Makefile for gfortran (Makefile.gfortran) for building the TEMPO executable to generate MP tables. So I added the variable MAKEFILE_SUFFIX to distinguish between intel and gfortran.

Btw, the code in src/core_atmosphere/utils/ related to TEMPO is outdated since the subroutine interfaces for the latest TEMPO are different than the ones used in the .F files in that directory. So another option for building the executable for TEMPO table generation is to update this directory (specifically atmphys_build_tables_tempo.F, build_tables_tempo.F, and Makefile) to match the latest TEMPO, but that is a job best suited for @AndersJensen-NOAA (and not necessary for this PR).

(cd physics_noaa/TEMPO; $(MAKE) -f Makefile.$(TEMPO_TBLS_MAKEFILE_SUFFIX) run_build_tables ; cp run_build_tables ../../../../../)

core_SMOKE: core_physics_init
(cd physics_noaa/SMOKE; cp ./MPAS/Makefile .; cp ./MPAS/mpas_smoke_wrapper.F90 .; $(MAKE) all)
Expand Down Expand Up @@ -290,6 +311,13 @@ clean:
( cd physics_noahmp/src; $(MAKE) clean )
( cd physics_noahmp/utility; $(MAKE) clean )
( if [ -d physics_noaa/TEMPO ]; then cd physics_noaa/TEMPO; $(MAKE) -f Makefile.mpas clean; fi )
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following is to clean the run_build_tables executable and related files generated by the new line above in this Makefile.

@# For cleaning the TEMPO executable for generating microphysics tables, call both
@# the intel and the gfortran Makefiles. This is because when the "clean" target
@# is invoked in upstream Makefiles, a compiler is not specified, so it is not
@# clear which of the two Makefiles to use here. By calling both, we ensure that
@# the directory is properly cleaned regardless of which compiler was used to build
@# the executable.
( if [ -d physics_noaa/TEMPO ]; then cd physics_noaa/TEMPO; $(MAKE) -f Makefile.gfortran clean; $(MAKE) -f Makefile.intel clean; fi )
( if [ -d physics_noaa/UGWP ]; then cd physics_noaa/UGWP; $(MAKE) clean; fi )
( if [ -d physics_noaa/RUCLSM ]; then cd physics_noaa/RUCLSM; $(MAKE) -f MPAS/Makefile clean; fi )
( if [ -d physics_noaa/MYNN-EDMF ]; then cd physics_noaa/MYNN-EDMF; cp ./MPAS/Makefile .; $(MAKE) clean; fi )
Expand Down
4 changes: 2 additions & 2 deletions src/core_atmosphere/physics/registry.chemistry.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,9 +199,9 @@
<!-- **************************************************************************************** -->

<streams>
Copy link
Copy Markdown
Author

@gsketefian gsketefian Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was necessary to avoid SMIOL errors apparently due to another stream named output defined in the main Registry.xml. This error causes the output .nc files to contain only the smoke variables defined here and none of the output variables for the output stream in the main Registry.xml. The output file is also renamed so two different streams are not writing to the same file.

<stream name="output"
<stream name="output_smoke"
type="output"
filename_template="history.$Y-$M-$D_$h.$m.$s.nc"
filename_template="history_smoke.$Y-$M-$D_$h.$m.$s.nc"
Copy link
Copy Markdown
Collaborator

@guoqing-noaa guoqing-noaa Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gsketefian This change will break the current rrfs-workflow.v2 (with smoke and dust) as discussed in #196 (comment)

We need to discuss this and have an agreement before making this change. Thanks!

output_interval="6:00:00"
runtime_format="separate_file">
<var name="aero_emis_for_enhmix"/>
Expand Down
1 change: 1 addition & 0 deletions src/core_atmosphere/stochastic_physics
Open in desktop
Submodule stochastic_physics added at 16d9ec
Loading