Feature/gdas phase1 mhs metop c configure by jianjunj · Pull Request #2116 · NOAA-EMC/GDASApp

jianjunj · 2026-04-08T20:49:09Z

Description

Adds UFO/JCB YAML configurations for MHS on Metop-B, Metop-C, and NOAA-19, reproducing GSI clear-sky observational errors and QC. Obs error counts and data passing QC are verified to match GSI output (tested at 00z, Feb 19, 2024).

Fix (this commit): Step H1 computed inflationFactor1 = inflationFactor0**(-0.5) with tpwc_index maxvalue: 1.0. At tpwc_index == 1.0, inflationFactor0 = 1 - 1² = 0, causing a division-by-zero before Step H2 could overwrite. Changed the H1 where bound to maxvalue: 0.9999999 (exclusive) so tpwc_index == 1.0 is handled solely by Step H2 (inflationFactor1 = 1.0). Applied to all three MHS templates.

# Before (division-by-zero at tpwc_index == 1.0)
  where:
  - variable:
      name: DerivedMetaData/tpwc_index
    maxvalue: 1.0

# After (exclusive upper bound; tpwc_index == 1.0 falls through to Step H2)
  where:
  - variable:
      name: DerivedMetaData/tpwc_index
    maxvalue: 0.9999999

Companion PRs

Issues

Automated CI tests to run in Global Workflow

…nicle.

…passively.

jianjunj · 2026-04-08T20:56:00Z

MHS_Metop-C

MHS_Meto-c

MHS_Meto-b.

MHS_Meto-b

MHS_N19 (channel 3 is passive in GSI)

MHS_N19(channel 3 is passive in GSI)

MHS observational errors and number of data passed QC in GSI are identical as those in UFO. Tested are done with GSI output at 00z, Feb 19, 2024.

Copilot

Pull request overview

This PR extends the JCB-GDAS observation configuration catalog by adding clear-sky CRTM/UFO YAML templates and corresponding observation-chronicle metadata for MHS radiance instruments (Metop-B, Metop-C, and NOAA-19), aiming to reproduce GSI-like observation errors and QC behavior.

Changes:

Added new MHS radiance observation templates (radiance_mhs_metop-b, radiance_mhs_metop-c, radiance_mhs_n19) with detailed QC/error inflation logic.
Added/updated MHS observation-chronicle metadata files to provide channel configuration values used by the templates.
Updated the atmosphere obs list to include the new radiance types (currently commented out).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
parm/jcb-gdas/observations/atmosphere/radiance_mhs_n19.yaml.j2	New clear-sky MHS(N19) CRTM/UFO obs config with QC + error modeling.
parm/jcb-gdas/observations/atmosphere/radiance_mhs_metop-c.yaml.j2	New clear-sky MHS(Metop-C) CRTM/UFO obs config with QC + error modeling.
parm/jcb-gdas/observations/atmosphere/radiance_mhs_metop-b.yaml.j2	New clear-sky MHS(Metop-B) CRTM/UFO obs config with QC + error modeling.
parm/jcb-gdas/observation_chronicle/atmosphere/radiance_mhs_n19.yaml	New MHS(N19) channel metadata (sim/active/bias/error/etc.).
parm/jcb-gdas/observation_chronicle/atmosphere/radiance_mhs_n18.yaml	New MHS(N18) channel metadata.
parm/jcb-gdas/observation_chronicle/atmosphere/radiance_mhs_metop-c.yaml	New MHS(Metop-C) channel metadata.
parm/jcb-gdas/observation_chronicle/atmosphere/radiance_mhs_metop-b.yaml	New MHS(Metop-B) channel metadata.
parm/jcb-gdas/observation_chronicle/atmosphere/radiance_mhs_metop-a.yaml	New MHS(Metop-A) channel metadata.
parm/atm/atm_obs_list.yaml.j2	Lists the new radiance types (commented out).

Copilot · 2026-04-08T20:56:43Z

+          exponents: [-0.5]
+    where:
+    - variable:
+        name: DerivedMetaData/tpwc_index
+      maxvalue: 1.0


inflationFactor1 is computed as inflationFactor0**(-0.5) while the preceding step allows tpwc_index up to (and including) 1.0, which makes inflationFactor0 = 1 - tpwc_index^2 equal to 0 at exactly 1.0. That yields 0**(-0.5) (division by zero / inf) before the later assignment overwrites it. Consider making the tpwc_index upper bound exclusive for the inflationFactor0/1 computations (or clamp with a small epsilon) and align the Step H2 comment/threshold accordingly.

@copilot apply changes based on this feedback

Fixed in commit 07e63cc. Changed Step H1's where condition from maxvalue: 1.0 to maxvalue: 0.9999999 in all three MHS templates (metop-b, metop-c, n19). This makes the upper bound exclusive so that when tpwc_index == 1.0 exactly (where inflationFactor0 = 0), only Step H2 runs and assigns inflationFactor1 = 1.0, avoiding the 0**(-0.5) division-by-zero. Also updated the Step H2 comment to accurately read tpwc_index >= 1.0.

Copilot · 2026-04-08T20:56:43Z

+          exponents: [-0.5]
+    where:
+    - variable:
+        name: DerivedMetaData/tpwc_index
+      maxvalue: 1.0


inflationFactor1 is computed as inflationFactor0**(-0.5) while the preceding step allows tpwc_index up to (and including) 1.0, which makes inflationFactor0 = 1 - tpwc_index^2 equal to 0 at exactly 1.0. That yields 0**(-0.5) (division by zero / inf) before the later assignment overwrites it. Consider making the tpwc_index upper bound exclusive for the inflationFactor0/1 computations (or clamp with a small epsilon) and align the Step H2 comment/threshold accordingly.

@copilot apply changes based on this feedback

Copilot · 2026-04-08T20:56:43Z

+          exponents: [-0.5]
+    where:
+    - variable:
+        name: DerivedMetaData/tpwc_index
+      maxvalue: 1.0


inflationFactor1 is computed as inflationFactor0**(-0.5) while the preceding step allows tpwc_index up to (and including) 1.0, which makes inflationFactor0 = 1 - tpwc_index^2 equal to 0 at exactly 1.0. That yields 0**(-0.5) (division by zero / inf) before the later assignment overwrites it. Consider making the tpwc_index upper bound exclusive for the inflationFactor0/1 computations (or clamp with a small epsilon) and align the Step H2 comment/threshold accordingly.

@copilot apply changes based on this feedback

Copilot · 2026-04-08T20:56:43Z

+  - filter: BlackList
+    where:
+    - variable:
+        name: DerivedMetaData/tpwc_index
+      minvalue: 1.0


This BlackList filter rejects entire observations when tpwc_index >= 1.0, but the preceding comment describes only tossing channels 3–5 based on O–F > 2K. If the intent is to only reject specific channels (or only for tpwc_index > 1.0), add filter variables/action (and/or make the 1.0 bound exclusive) so the behavior matches the comment and avoids the current overlap with the subsequent tpwc_index maxvalue: 1.0 check.

Suggested change

- filter: BlackList

where:

- variable:

name: DerivedMetaData/tpwc_index

minvalue: 1.0

- filter: BlackList

filter variables:

- name: brightnessTemperature

channels: 3-5

where:

- variable:

name: DerivedMetaData/tpwc_index

minvalue: 1.0

action:

name: reject

Copilot · 2026-04-08T20:56:44Z

+    where:
+    - variable:
+        name: DerivedMetaData/tpwc_index
+      minvalue: 1.0


This BlackList filter rejects entire observations when tpwc_index >= 1.0, but the preceding comment describes only tossing channels 3–5 based on O–F > 2K. If the intent is to only reject specific channels (or only for tpwc_index > 1.0), add filter variables/action (and/or make the 1.0 bound exclusive) so the behavior matches the comment and avoids the current overlap with the subsequent tpwc_index maxvalue: 1.0 check.

Suggested change

where:

- variable:

name: DerivedMetaData/tpwc_index

minvalue: 1.0

filter variables:

- name: brightnessTemperature

channels: 3-5

action:

name: reject

where:

- variable:

name: DerivedMetaData/tpwc_index

minvalue: 1.000001

Copilot · 2026-04-08T20:56:44Z

+  # Step K2: Toss channel 3-5 data with O-F > 2K 
+  - filter: BlackList
+    where:
+    - variable:
+        name: DerivedMetaData/tpwc_index
+      minvalue: 1.0
+  - filter: Background Check
+    filter variables:
+    - name: brightnessTemperature
+      channels: 3-5
+    absolute threshold: 2.0
+    where:
+    - variable:
+        name: DerivedMetaData/tpwc_index
+      maxvalue: 1.0


This BlackList filter rejects entire observations when tpwc_index >= 1.0, but the preceding comment describes only tossing channels 3–5 based on O–F > 2K. If the intent is to only reject specific channels (or only for tpwc_index > 1.0), add filter variables/action (and/or make the 1.0 bound exclusive) so the behavior matches the comment and avoids the current overlap with the subsequent tpwc_index maxvalue: 1.0 check.

Suggested change

# Step K2: Toss channel 3-5 data with O-F > 2K

- filter: BlackList

where:

- variable:

name: DerivedMetaData/tpwc_index

minvalue: 1.0

- filter: Background Check

filter variables:

- name: brightnessTemperature

channels: 3-5

absolute threshold: 2.0

where:

- variable:

name: DerivedMetaData/tpwc_index

maxvalue: 1.0

# Step K2: Toss channel 3-5 data with O-F > 2K

- filter: Background Check

filter variables:

- name: brightnessTemperature

channels: 3-5

absolute threshold: 2.0

…n MHS templates Agent-Logs-Url: https://github.com/NOAA-EMC/GDASApp/sessions/499257a3-8036-47ab-8c43-8b7bc6e3dfab Co-authored-by: jianjunj <183659102+jianjunj@users.noreply.github.com>

jianjunj added 15 commits February 11, 2026 10:50

Added satinfo for MHS (N19, metop-b, and metop-c) in observation_chro…

823a8d7

…nicle.

Added satinfo for MHS (N18 and metop-a) in observation_chronicle.

fc8517e

Initial configurations of MHS_Metop-C observations.

0e42c21

Correct the variable 'DerivedMetaData/surface_water_ice_snow'.

17d9e10

Corrected functions for cloud_index and tpwc_index.

4b4dd45

Added error inflation and more QC in MHS data.

009b1da

Upate QC in MHS data/

aac869f

Updated the final background check.

fd07a6a

Add 'ObsErrorBound'.

13a8ce0

Updated observational error inflations.

00e7446

Set passedBenchmark: 0.

a9a0127

Add 'use passive_bc: true' in the InterChannelConsistencyCheck.

9ec21f2

Add configurations for mhs_metop-b and mhs_n19.

881fec7

Added atmospheric observations mhs_metop-b, mhs_metop-c, and mhs_n19 …

e433664

…passively.

Merge branch 'develop' into feature/gdas_phase1_MHS_Metop-C_configure

88fc8c9

jianjunj requested a review from Copilot April 8, 2026 20:50

jianjunj self-assigned this Apr 8, 2026

Copilot started reviewing on behalf of jianjunj April 8, 2026 20:50 View session

jianjunj requested a review from ADCollard April 8, 2026 20:51

Copilot AI reviewed Apr 8, 2026

View reviewed changes

Copilot started work on behalf of jianjunj April 8, 2026 21:05 View session

Fix division-by-zero in inflationFactor1 when tpwc_index equals 1.0 i…

07e63cc

…n MHS templates Agent-Logs-Url: https://github.com/NOAA-EMC/GDASApp/sessions/499257a3-8036-47ab-8c43-8b7bc6e3dfab Co-authored-by: jianjunj <183659102+jianjunj@users.noreply.github.com>

Copilot finished work on behalf of jianjunj April 8, 2026 21:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/gdas phase1 mhs metop c configure#2116

Feature/gdas phase1 mhs metop c configure#2116
jianjunj wants to merge 16 commits intodevelopfrom
feature/gdas_phase1_MHS_Metop-C_configure

jianjunj commented Apr 8, 2026 •

edited by Copilot AI

Loading

Uh oh!

jianjunj commented Apr 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 8, 2026

Uh oh!

jianjunj Apr 8, 2026

Uh oh!

Copilot AI Apr 8, 2026

Uh oh!

Copilot AI Apr 8, 2026

Uh oh!

jianjunj Apr 8, 2026

Uh oh!

Copilot AI Apr 8, 2026

Uh oh!

jianjunj Apr 8, 2026

Uh oh!

Copilot AI Apr 8, 2026

Uh oh!

Copilot AI Apr 8, 2026

Uh oh!

Copilot AI Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jianjunj commented Apr 8, 2026 • edited by Copilot AI Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Companion PRs

Issues

Automated CI tests to run in Global Workflow

Uh oh!

jianjunj commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

jianjunj Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

jianjunj Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

jianjunj Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jianjunj commented Apr 8, 2026 •

edited by Copilot AI

Loading

jianjunj commented Apr 8, 2026 •

edited

Loading