On-the-fly nested-atmosphere boundary conditions + Davies relaxation#388
Open
glwagner wants to merge 73 commits into
Open
On-the-fly nested-atmosphere boundary conditions + Davies relaxation#388glwagner wants to merge 73 commits into
glwagner wants to merge 73 commits into
Conversation
…ognostic maps Extracts air_density / virtual_temperature / potential_temperature / liquid_ice_potential_temperature / total_water_specific_humidity as @inline scalar maps (GPU-safe), so the same definitions serve both breeze_prognostic_state (broadcast over Fields) and the upcoming on-the-fly nesting boundary conditions (per boundary-face node). Renames qᶜ/qⁱ → qᶜˡ/qᶜⁱ. Prep for the state-derived nesting BC; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ng boundary value Internal carrier (not user-facing): holds a NamedTuple of parent-state FieldTimeSeries + per-source interpolation transforms (log for p/T, identity otherwise) + a prognostic map. getbc interpolates each source at the boundary-face node and applies the map → a child prognostic computed on the fly. Interpolation uses the #5726-style interpolate(func,…) (func-space, no inverse) and un-maps locally via inverse_transform (log→exp, identity→identity), so log-space pressure interpolation is exp(blend of log p) — faithful ln-p. Reuses the node/boundary-index/clock helpers. The model-specific maps (Breeze ρ/ρu/ρθˡⁱ/ρqᵉ) come next, in the ext. CPU smokes: a 2-field transform reconstructs the right face values; a log-space single source recovers the geometric interpolation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The on-the-fly nesting BC (ParentStateBoundary) uses Oceananigans' interpolate(f, …) (merged in CliMA/Oceananigans.jl#5726) for log-space interpolation of parent state, so the local shim was dropped (in #386). Point Oceananigans at main via [sources] until a release containing #5726 is tagged, then bump [compat] and drop [sources]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AirDensity / MomentumDensity / EnergyDensity / MoistureDensity: isbits callables carrying ThermodynamicConstants that map an interpolated parent-state NamedTuple to one CompressibleDynamics density-weighted prognostic, reusing the pointwise maps. These are the ParentStateBoundary 'transform' for each child prognostic (ρ, ρu/ρv/ρw, ρθ, ρqᵗ), replacing the materialized breeze_prognostic_state parent series. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… the fly Add nested_lateral_boundary_conditions(parent, constants, moisture_name): for each child prognostic (ρ, ρu, ρv, ρe, <moisture>) build a ParentStateBoundary over the parent's raw sources (velocity, T, qᵛ, qᶜˡ, qᶜⁱ, p), interpolating p/T in log space and applying the matching density-weighted transform. ρu/ρv use NormalFlowBC, ρ/ρe/moisture use ValueBC. atmosphere_simulation gains a `parent` kwarg: when given, these lateral BCs are merged (per-side) with the coupling bottom-flux BCs. Workflow: parent -> child(parent) -> NestedModel. Replaces the materialized breeze_prognostic_state parent series for the lateral BCs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…osphere - ParentStateTarget (NestedSimulations): the interior-relaxation analogue of ParentStateBoundary — a Relaxation target(x,y,z,t) that interpolates parent state at the node and applies the prognostic transform (reuses _query_source). - Specific transforms LiquidIcePotentialTemperature / TotalWater for the Davies targets (Breeze SpecificForcing applies the rho-weight at kernel time). - nested_relaxation_forcings(parent_atmosphere, constants; rate, mask): forcings keyed (u, v, theta, q_e); u/v relax toward the raw parent velocities, theta/q_e toward on-the-fly specific targets. Replaces the materialized breeze_prognostic_state targets. - atmosphere_simulation gains relaxation_rate / relaxation_mask; with parent_atmosphere it merges Davies forcings into `forcing`. Renamed the `parent` kwarg/arg to `parent_atmosphere` (avoids shadowing Base.parent / Oceananigans parent(field)). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
parent_state_boundary.jl imported instantiated_location from Oceananigans.Fields but uses the qualified Oceananigans.instantiated_location, so the import was both stale (unused) and not-via-owner (owner is Oceananigans). Remove it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment out the other examples so the docs build exercises only breeze_downscaling_era5 (the on-the-fly nesting BC case); restore before merge. Add it via the 'build all examples' label (NUMERICAL_EARTH_BUILD_ALL_EXAMPLES=true). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…_grid; ...) Per the cleaner API: build the child and the NestedModel together rather than threading the parent through atmosphere_simulation. The relaxation rate/mask are now nesting-level config (on nested_atmosphere_model) yet still baked into the child's forcing at construction (tendency-integrated, no rebuild/step-nudge). - atmosphere_simulation reverts to the generic child builder (drop parent_atmosphere / relaxation_rate / relaxation_mask kwargs). - New `nested_atmosphere_model(parent_atmosphere, child_grid; relaxation_rate, relaxation_mask, child_kw...)` (declared in NestedSimulations, method in the Breeze extension): derives the lateral BCs + Davies forcings from the parent on the fly, hands them to atmosphere_simulation as boundary_conditions/forcing (per-side merge), and wraps in NestedModel(parent, child). - Exported nested_atmosphere_model from NumericalEarth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Establishes the rule (AGENTS.md): never unpack a property right after a constructor (foo(...).model / .child) — provide a constructor that returns what's needed. - atmosphere_model(grid; ...) returns the Breeze AtmosphereModel. - atmosphere_simulation(grid; Δt, kw...) = Simulation(atmosphere_model(grid; kw...); Δt). - nested_atmosphere_model builds the child via atmosphere_model (no .model unpack) and returns NestedModel(parent, child) — used directly (no .child unpack). - Declared/exported atmosphere_model from Atmospheres / NumericalEarth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enabling layer for the example rewrite onto the nested_atmosphere_model API: - NestedModel forwards set!(nm; kwargs...) to the child, so the IC can be set on the nested model directly (no .child unpack). - Export nested_lateral_boundary_conditions (stub in NestedSimulations, method in the Breeze ext) so a dynamical-init twin can share the parent-derived lateral BCs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse the ~125-line parent-derivation block (parent grid + materialized parent_series + per-snapshot breeze_prognostic_state loop + hydrostatic-pressure integration) into a single `ERA5PrescribedAtmosphere(region, dates; …)`, and drive the child through `nested_atmosphere_model(parent, grid; relaxation_rate, relaxation_mask, …)` — lateral BCs + Davies relaxation are now derived on the fly from the parent's raw state, no materialized parent prognostic series. - model: `atmosphere_simulation(grid; …).model` → `nested_atmosphere_model(parent, grid; …)` returning a `NestedModel` (no `.model`/`.child` unpack); run via `Simulation(model)`. - surface BCs: pass bottom-only `FieldBoundaryConditions`, merged per-side with the parent-derived lateral BCs inside `nested_atmosphere_model`. - reference_θ: domain-mean θˡⁱ of the child IC (computed once, reused by `set!`), not the (removed) parent_series. - DFI twin: `atmosphere_model(grid; …)` + shared `nested_lateral_boundary_conditions`. - animation parent row: reconstruct (ρ, θˡⁱ, qᵗ) on the fly via breeze_prognostic_state on the parent's native PressureLevelGrid, sampled with the same `cut_plane`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…gans main) Oceananigans main renamed `NormalFlowBoundaryCondition` to `OpenBoundaryCondition` (classification `Open`, scheme `PerturbationAdvection`). Since this branch depends on Oceananigans main (the `interpolate(f, …)` [sources] pin), the old name no longer exists: Julia 1.12 emits only a "binding undeclared at import time" warning at precompile, but any call site (the nested lateral-BC builder) would throw at runtime — i.e. the example's nested_atmosphere_model path. Rename all imports, the OpenBoundaryCondition(condition; scheme) calls, and stale comments. Verified: precompiles with the warning gone and loads cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…Oceananigans main)" This reverts commit 5a351d2.
…nscaling_era5
The example (built in docs CI under the "build all examples" label) does
`using CloudMicrophysics` (OneMomentCloudMicrophysics via BreezeCloudMicrophysicsExt),
`using JLD2` (jldsave), and `using NaturalEarth` — none were in docs/Project.toml, so the
docs build failed at `using CloudMicrophysics` ("Package not found in current path").
Verified locally: docs resolves cleanly with the three added and BreezeCloudMicrophysicsExt
loads (OneMomentCloudMicrophysics accessible).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the GeoInterface/NaturalEarth border-flattening out of the breeze_downscaling_era5 example into an extension. `natural_earth_lines(name; scale=50)` returns NaN-separated (lons, lats) vectors for a Natural Earth feature layer, ready for `lines!`. Backend-agnostic (no Makie dependency) — drawing stays in the caller. - Core: `function natural_earth_lines end` stub + export in NumericalEarth.jl. - ext/NumericalEarthNaturalEarthExt.jl: triggered by ["NaturalEarth", "GeoInterface"] (using NaturalEarth pulls GeoInterface transitively, so it fires on `using NaturalEarth`). - Project.toml: NaturalEarth + GeoInterface weakdeps/extensions/compat. - Example: drop `import GeoInterface as GI` + the inline append_border!/natural_earth_lines block; call the exported function. docs/Project.toml gains GeoInterface. Verified locally (docs project): the ext loads and natural_earth_lines returns the US state + country border vectors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
construct_native_grid mapped the bbox into the dataset's native longitude convention and labeled the grid there, dropping the caller's convention. For ERA5 ([0,360] native) driven from a [-180,180] bbox (e.g. a US-centered nest), the parent grid came out in [0,360] while the child LAM is [-180,180], so the on-the-fly nesting BCs failed the "source brackets child" check in longitude. Relabel the grid longitudes back to the bbox convention (shift is 0 when they already match, ±360 when they differ; data is array-indexed so ordering is unchanged). Verified on a GPU ERA5→Breeze nest: the longitude bracket now passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…'s model build nested_atmosphere_model now supplies overridable defaults for a nested limited-area atmosphere: - microphysics = default_nested_microphysics() — 1-moment mixed-phase when Breeze's CloudMicrophysics extension is loaded (resolved at call time via Base.get_extension), else SaturationAdjustment - momentum_advection = WENO(order = 9) - coriolis = SphericalCoriolis() - a compressible split-explicit `dynamics` (UpperSponge over the top `damping_depth` m at `damping_rate`, no divergence damping) built from `surface_pressure`/`reference_potential_temperature`, plus a matching ρw Rayleigh lid sponge added to `forcing`. The example's model build drops to the IC-derived args (surface_pressure, reference_potential_temperature, relaxation, surface BCs); the DFI twin takes its moisture key from `model.microphysics`. Construction validated on CPU (the defaults build a NestedModel; ext-detected microphysics resolves correctly). Note: set_to_mean!/compute_reference_state recompute only the anelastic ReferenceState, not the split-explicit ExnerReferenceState used here, so the perturbation reference is set explicitly via reference_potential_temperature (reference_θ). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ρ = p / (Rᵐ T) on the grid of `temperature`, with the moist mixture gas constant Rᵐ = qᵈ Rᵈ + qᵛ Rᵛ, qᵈ = 1 − qᵛ − qᶜ − qⁱ — the same EOS as hydrostatic_pressure_from_surface, so a density built from that pressure is mutually consistent. Optional qᵛ/qᶜ/qⁱ (dry if omitted); returns a CenterField. A focused utility for initializing a compressible model's density from an analysis temperature + (hydrostatic) pressure: `set!(model; ρ = density_from_pressure(T, p; …))`, replacing the IC use of breeze_prognostic_state. Verified: dry/moist values match p/(Rᵐ T). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…on't bracket) The on-the-fly nesting BCs require the parent to bracket the child in x/y, but the child legitimately extends below the parent's lowest level in z (ERA5 pressure-level data doesn't reach the surface) — the strict z-bracket check (inherited from Relaxation-on-FTS) wrongly rejected this, blocking the ERA5→Breeze example. Two changes: - validate_source_bracket: enforce bracketing in x/y only (a too-small horizontal parent region is a real error); the vertical is allowed to be exceeded. - clamp_to_source_z: clamp the query's z into the source's center-z range before interpolating, so an out-of-range child node returns the parent's edge value (constant extrapolation). This is explicit and deterministic — relying on Oceananigans' out-of-grid interpolation was halo-dependent and gave the wrong value above the top level (a test showed the bottom clamped correctly but the top didn't). Test (CPU): a parent T linear in z, child spanning below and above; in-range nodes recover T exactly, out-of-range nodes return the clamped edge value (102 below, 108 above). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…raction - breeze_nested_atmosphere: extract parent_state_fields helper shared by nested_lateral_boundary_conditions and nested_relaxation_forcings - default_nested_dynamics: drop the redundant reference_potential_temperature guard (CompressibleDynamics stores nothing unconverted); guard only surface_pressure - hydrostatic_pressure: factor the shared interior temperature+moisture array extraction into interior_temperature_and_moisture, used by both hydrostatic_pressure_from_surface and density_from_pressure Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The breeze_downscaling_era5 docs example builds the nested model on a terrain-following LatitudeLongitudeGrid on the GPU, which exposed three host-side scalar-indexing / GPU-IR failures in the nesting machinery: - lid_sponge_mask: znode(1,1,Nz+1, grid, …) reads the device z-array on a terrain-following grid; compute it on a CPU mirror of the grid (as Breeze's reference-state build does) so the one-off top-of-domain lookup doesn't scalar-index a GPU array. - nested_relaxation_forcings: the u/v Davies targets were raw parent FieldTimeSeries, taking Oceananigans' FTS-target Relaxation path whose validate_fts_target_extent does extrema(znodes(grid)) — scalar-indexing the child's 3D terrain-following z-nodes on GPU. Route u/v through ParentStateTarget like θ/qᵉ; this also z-clamps to the parent range instead of reading unfilled FTS halos. - _parent_state / ParentStateTarget: mapping a closure over the (sources, interpolations) NamedTuples is a dynamic invocation rejected by GPU IR validation. Interpolate via explicit tuple recursion (interpolate_sources), rebuilding the state NamedTuple from compile-time keys. Validated on GPU: the nested model now builds successfully. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…Breeze 0.7)
Rework the on-the-fly nested relaxation/lid-sponge as concrete per-variable discrete
forcings and simplify breeze_downscaling_era5.jl. NOTE: GPU time-stepping is blocked by
in-kernel interpolation of the parent ERA5 FieldTimeSeries (DatasetBackend) on the
PressureLevelGrid — InvalidIRError (gpu_gc_pool_alloc + throw_methoderror). Model build,
IC, and all tendency kernels compile when that interpolate is stubbed.
ext (breeze_nested_atmosphere.jl):
- Replace ParentStateTarget relaxation with concrete `<: NestedForcing` callables
(PotentialTemperatureRelaxation, MoistureRelaxation, U/VRelaxation, LidSponge), each a
discrete `(i,j,k,grid,clock,fields)` returning the full ρᵈ-weighted tendency.
- Bypass ContinuousForcing/DiscreteForcing via
`materialize_atmosphere_model_forcing(::NestedForcing) = f` + `is_density_tendency_forcing`.
- Add `terrain` kwarg (materialize_terrain! before model build); lid-sponge znode via @allowscalar.
src (parent_state_boundary.jl): rename clamp_to_source_z → z_clamp; remove ParentStateTarget.
example: terrain into constructor; IC below model build; ρ via density_from_pressure; T set
directly with compute_reference_state=true (drops reference_θ); BoundingBox auto-snap (drop
snap_out); Field{Center,Center,Nothing} orography; type-stable lateral_mask (oftype).
Project/docs/test .toml: temporary [sources] pin Breeze → glw/set-reference-state (0.7) for
compute_reference_state (#812); compat 0.7.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lw/nested-atmosphere-bc # Conflicts: # ext/NumericalEarthBreezeExt/breeze_nested_atmosphere.jl # src/EarthSystemModels/NestedSimulations/parent_state_boundary.jl
…variable types
Replace the generic ParentStateBoundary (sources/interpolations/transform + interpolate_sources)
with concrete per-variable lateral-BC samplers, mirroring the discrete relaxation forcings. Both
layers now share one set of parent-state samplers.
- src/parent_state_boundary.jl: thin `ParentBoundary{Dim,Side,LX,LY,LZ, V, G}` carrier wrapping a
concrete `value(X,t)` sampler; getbc supplies the boundary-face node, sampler z_clamps + interpolates.
Removes ParentStateBoundary/ParentStateTarget/interpolate_sources/inverse_transform/_query_source.
- ext: shared `air_density_at`/`liquid_ice_θ_at`/`total_water_at`/`velocity_at` samplers (interpolate
raw parent fields + combine, log-space T/p); concrete BC samplers AirDensityBoundary/
MomentumDensityBoundary/EnergyDensityBoundary/MoistureDensityBoundary; relaxation forcings
refactored onto the same samplers (store `constants`); `tuple(elem)` notation; `field_name` (no `_`).
Note: still GPU-WIP — the in-kernel parent-FTS interpolate blocker is unresolved (next).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- metadata FTS: match `times` to the grid float type. Float64 times on a Float32
grid made `interpolate`'s time weight Float64, so the value was Union{Float32,
Float64} — boxed inside GPU tendency/halo kernels (the InvalidIRError root cause).
- ERA5 `pressure_level_field`: drop the redundant `.* 100`. `ERA5_all_pressure_levels`
is already Pa (via the `hPa` factor), so the field was 100× too high ([1e4, 1e7] Pa),
making sampled θ unphysical (~85 K).
- nested samplers interpolate linearly (`identity`) for all parent fields; thread the
potential-temperature reference pressure pˢᵗ from the child `dynamics.standard_pressure`
instead of a hardcoded 1e5 (which also leaked Float64 into the combine).
- remove dead prognostic-transform structs (superseded by the discrete samplers).
Validated on CPU: sampler is Body::Float32 and returns a physical θ (~318 K).
Requires the Oceananigans method `interpolate(func, X, Time, ::AbstractField, …)`
(drops the time index for a static `Field` — e.g. the pressure-level coordinate);
pending upstream.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… on the parent grid Replace the per-node, in-kernel thermodynamic combine (which never compiled on GPU) with a precomputed derived-state design that compiles + steps cleanly on GPU (validated on an H100). - StateExchanger (ext/breeze_state_exchanger.jl): a fused `_compute_child_prognostics!` kernel computes the child's Breeze prognostics — dry density ρᵈ = ρ(1−qᵗ), dry-weighted ρθ/ρu/ρv, and total-weighted ρqᵛ — from the raw ERA5 state ON THE PARENT GRID, stored as FieldTimeSeries. Density weighting matches Breeze `establish_densities!`/`set!`. Condensate inputs are configurable (parent / another source / ZeroField ⇒ qᵗ = qᵛ). `exchange_state!` refreshes them from the parent. - NestedModel gains an `exchanger` field; `time_step!(nested, Δt)` / `update_state!` call `exchange_state!(exchanger, clock.time)` before the child steps, so the child's boundary conditions and forcings see current parent-derived prognostics. - nested_atmosphere_model now builds those FTS and drives the child through the generic FTS machinery: `parent_boundary_conditions` (Interpolated BCs; energy keyed :ρe) + `parent_forcings` (Relaxation). The tendency/halo kernels only interpolate a plain FieldTimeSeries — no combine, no z_clamp, no MappedData — so they specialize on the same type Breeze compiles for any FTS forcing. Removed the per-node samplers, the NestedForcing bypass, and the now-unused nested_lateral_boundary_conditions. Requires (Oceananigans, pending upstream): FTS `Relaxation` vertical-clamp (`validate_fts_target_extent` horizontal-only), mirroring the Interpolated BC — a nested child extends below the parent's lowest level. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…he convective layer) Default damping_depth was a fixed 3 km, too thin — vertically-propagating gravity/acoustic waves reflected off the rigid top and grew. Default it instead to default_lid_depth(grid) = grid.Lz/4 (~5 km for a ~19.5 km column, sponge base ~15 km): deep enough to absorb genuine top reflection now that the consistent exchanger-derived IC removes the wall-mode source, yet thin enough to leave the deep-convective layer (~12–16 km) undamped. Uses the grid's vertical extent Lz — not the top node height, which on a terrain-following grid carries the terrain offset — so it scales with any model top. Lifts both the dynamics UpperSponge and the ρw Relaxation. CPU smoke on defaults (no kwargs): max|w| stays bounded (~4–6 m/s); the remaining ~6 m/s is a lateral wall residual (ρᵈ relaxation, tracked separately), not a top runaway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
C2 (thin Lz/4 sponge, defaults-only) showed max|w| creeping to ~6 from a persistent LATERAL-WALL residual (viz: edge striping up the column), which a thin top sponge can't damp and the old thick sponge only masked. Root cause is the un-relaxed near-wall mass/density field: the Davies zone relaxed ρθ/ρu/ρv/ moisture but NOT the density. Add ρᵈ = prognostic.ρᵈ to the relaxed var set, matching WRF (nudges dry mass μ) and MPAS (nudges ρ). parent_forcings already handles any var set; ρᵈ is the compressible-core density prognostic, so the Relaxation targets it directly. Expected: max|w| tightens back toward ~4 with a thin (convection-protecting) sponge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A rebase re-introduced `include("parent_state_boundary.jl")` into
NestedModels.jl while the file itself stayed deleted (from fdf6e63),
leaving HEAD unloadable (include of a missing file). Remove the stray
include so NumericalEarth loads again; ParentBoundary stays gone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents the 12 h convection-permitting downscaling runs across the 12→0.5 km resolution sweep (gpu-prod H100s + cpu partition): the validated stable configuration (option-A exchanger IC, domain-scaled Lz/4 lid sponge, per-side momentum BCs + ρᵈ Davies relaxation, WENO-5 momentum + bounded per-tracer moisture advection, no explicit closure, adaptive-Δt wizard), the fixed-domain resolution refinement (one shared ERA5 region), the compute allocation, the animation/slice outputs, and the harness/batch/launcher files in scratch_runenv/. Includes the explicit-nudge over-relaxation constraint (rΔt ≲ 2) motivating the Δt-scaled Davies rate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three CI failures, all in our nested-atmosphere code: - `fill_halo_regions!` was imported from Oceananigans.Fields in breeze_state_exchanger.jl; its owner (Base.which) is Oceananigans.BoundaryConditions. Import it from there so check_all_explicit_imports_via_owners passes. - NestedModels carried stale `znode`/`Center` imports (in interpolated_fts_boundary.jl) left over from the removed parent_state_boundary.jl. Drop them. - The per-side bc_types refactor dropped the guard that rejected a `schemes` entry for a field that is NormalFlow on no side (the scheme would be silently ignored). Restore it, adapted to the per-side design: error only when none of the field's sides is a NormalFlowBoundaryCondition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ewquon
reviewed
Jul 3, 2026
Comment on lines
+43
to
+55
| # Cosine-ramp Davies mask: 1 at the lateral walls, ramping to 0 over the outermost `width` cells. | ||
| # Captures plain numbers so the closure is GPU-compilable. | ||
| function davies_relaxation_mask(grid, width) | ||
| λ₁, λ₂ = extrema(λnodes(grid, Face(), Center(), Center())) | ||
| φ₁, φ₂ = extrema(φnodes(grid, Center(), Face(), Center())) | ||
| Nx, Ny, _ = size(grid) | ||
| w = width * max((λ₂ - λ₁) / Nx, (φ₂ - φ₁) / Ny) | ||
| return (λ, φ, z) -> begin | ||
| d = min(λ - λ₁, λ₂ - λ, φ - φ₁, φ₂ - φ) | ||
| d >= w && return zero(λ) | ||
| return oftype(λ, (1 + cos(π * d / w)) / 2) | ||
| end | ||
| end |
Collaborator
There was a problem hiding this comment.
Suggested change
| # Cosine-ramp Davies mask: 1 at the lateral walls, ramping to 0 over the outermost `width` cells. | |
| # Captures plain numbers so the closure is GPU-compilable. | |
| function davies_relaxation_mask(grid, width) | |
| λ₁, λ₂ = extrema(λnodes(grid, Face(), Center(), Center())) | |
| φ₁, φ₂ = extrema(φnodes(grid, Center(), Face(), Center())) | |
| Nx, Ny, _ = size(grid) | |
| w = width * max((λ₂ - λ₁) / Nx, (φ₂ - φ₁) / Ny) | |
| return (λ, φ, z) -> begin | |
| d = min(λ - λ₁, λ₂ - λ, φ - φ₁, φ₂ - φ) | |
| d >= w && return zero(λ) | |
| return oftype(λ, (1 + cos(π * d / w)) / 2) | |
| end | |
| end | |
| # Ramp shapes (isbits callables) for a nudging zone: weight vs. normalized distance from the wall, s ∈ [0, 1]. | |
| # Contract: ramp(0)=1, ramp(1)=0, monotone between. | |
| struct CosineRamp end | |
| struct SmoothStepRamp end | |
| @inline (::CosineRamp)(s) = (1 + cos(π * s)) / 2 | |
| @inline (::SmoothStepRamp)(s) = 1 - s^2 * (3 - 2s) | |
| # Davies mask: 1 at the lateral walls, ramping to 0 over the outermost `width` cells. | |
| function davies_relaxation_mask(grid, width; ramp = CosineRamp()) | |
| λ₁, λ₂ = extrema(λnodes(grid, Face(), Center(), Center())) | |
| φ₁, φ₂ = extrema(φnodes(grid, Center(), Face(), Center())) | |
| Nx, Ny, _ = size(grid) | |
| w = width * max((λ₂ - λ₁) / Nx, (φ₂ - φ₁) / Ny) | |
| return (λ, φ, z) -> begin | |
| d = min(λ - λ₁, λ₂ - λ, φ - φ₁, φ₂ - φ) | |
| s = clamp(d / w, zero(d), one(d)) | |
| return oftype(d, ramp(s)) | |
| end | |
| end |
Is now a good time to change the ramping function to something more computationally efficient?
Member
Author
There was a problem hiding this comment.
I think so! I doubt the precise structure of the ramping matters? So efficiency is good.
ewquon
reviewed
Jul 3, 2026
Comment on lines
+60
to
+64
| function lid_sponge_mask(grid, depth) | ||
| z_top = @allowscalar znode(1, 1, size(grid, 3) + 1, grid, Center(), Center(), Face()) | ||
| d = convert(eltype(grid), depth) | ||
| return (λ, φ, z) -> (s = clamp((z - (z_top - d)) / d, zero(z), one(z)); s * s * (3 - 2s)) | ||
| end |
Collaborator
There was a problem hiding this comment.
Suggested change
| function lid_sponge_mask(grid, depth) | |
| z_top = @allowscalar znode(1, 1, size(grid, 3) + 1, grid, Center(), Center(), Face()) | |
| d = convert(eltype(grid), depth) | |
| return (λ, φ, z) -> (s = clamp((z - (z_top - d)) / d, zero(z), one(z)); s * s * (3 - 2s)) | |
| end | |
| function lid_sponge_mask(grid, depth; ramp = SmoothStepRamp()) | |
| z_top = @allowscalar znode(1, 1, size(grid, 3) + 1, grid, Center(), Center(), Face()) | |
| d = convert(eltype(grid), depth) | |
| return (λ, φ, z) -> (s = clamp((z - (z_top - d)) / d, zero(z), one(z)); ramp(s)) | |
| end |
with the change above
On Breeze ≥0.7 (#812), a bare `set!(model; θ, u)` no longer derives the prognostic density from the reference state — the returned model carried dry_density ≡ total_density ≡ 0. Anything that divides by density then NaNs; concretely the MOST surface-flux coupling (AtmosphereOceanModel / AtmosphereLandModel) produced NaN stress and sensible heat. Initialize the model to a resting, hydrostatically balanced state at the reference potential temperature via `set!(model; θ, ρ = HydrostaticallyBalancedDensity(; surface_pressure))`, so the returned atmosphere has a valid density. A later user `set!` of θ/velocities (without a `ρ` entry) preserves it. Verified: dry/total density now 0.45–1.20 (was 0), momentum ρu = ρᵈ·u (was 0), and the coupling fluxes are finite (test_breeze_coupling Construction 7/7 and the MOST-stress feedback tests pass). The nested child path is unaffected — initialize_nested_child! fully overwrites this state with the ERA5-derived IC. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ewquon
reviewed
Jul 3, 2026
| - **Advection (implicit LES, no explicit closure):** `momentum_advection = WENO(order=5)`; per-tracer | ||
| `scalar_advection = (ρθ = WENO(5), ρqᵉ = WENO(5, bounds=(0,1)), ρqʳ = …, ρqˢ = …)` — moisture/precip | ||
| are positivity-bounded, `ρθ` must **not** be bounded to `[0,1]`; `closure = nothing`. | ||
| - **Time stepping:** adaptive `conjure_time_step_wizard!(cfl = 0.7, max_Δt = 20 s)` on the (slower) |
Collaborator
There was a problem hiding this comment.
max dt = 20 s is pretty conservative (small) for dx ~ 12 km
On Breeze 0.7 the coupled compressible solver is unstable at the test's Δt = 10 s (max|w| blows up → θ^γ DomainError); the stable Δt dropped below 5 s, whereas Breeze 0.6 ran this stably at Δt = 10. The DFI balancer that would settle the initial state also crashes on SaturationAdjustment (FieldError: no field qˡ). Both are tracked upstream in NumericalEarth/Breeze.jl#827. Wrap the `Time stepping` and `SST responds to fluxes` subtests so the run! is guarded and the outcome is `@test_broken` — recording them as broken (not suite errors) and flipping to an error (prompting re-enable) once the run completes cleanly again. The MOST-stress AtmosphereLandModel subtest passes (6/6) with the atmosphere_model density-init fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
examples/breeze_downscaling_era5.jl `using CopernicusClimateDataStore` (activates the ERA5 download extension) but the package was missing from the docs environment, so the docs build errored "Package CopernicusClimateDataStore not found". The example is build_always=false (built only under the "build all examples" label), so normal docs CI is unaffected; this lets the labeled build resolve. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ewquon
reviewed
Jul 3, 2026
ewquon
reviewed
Jul 3, 2026
Comment on lines
+18
to
+19
| # ρθ = ρᵈ · θˡⁱ, ρu = ρᵈ · u, ρv = ρᵈ · v ← DRY-weighted (energy + momentum) | ||
| # ρqᵛ = ρ · qᵛ ← TOTAL-weighted (moisture mass density) |
Collaborator
There was a problem hiding this comment.
Inconsistent use of ρ vs ρᵈ here.
Member
Author
There was a problem hiding this comment.
I think the dycore does actually evolve ρ · qᵛ = ρᵛ so this may be correct?
ewquon
reviewed
Jul 3, 2026
ewquon
reviewed
Jul 3, 2026
ewquon
approved these changes
Jul 3, 2026
…lization plan Compute allocation moves to the gpua100x4 (4×A100-40GB) and gpua100 partitions with gpuprod H100 nodes reserved for CPU-side rendering; adds a Visualization section (per-run CairoMakie render jobs chained via afterok, mp4 + final panel, cross-run gallery) and documents the recreated scratch_runenv (now gitignored). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gpua100 node failed two consecutive boot probes (16+ min silent vs gpuprod's 1m35s spin-up), so the fifth sweep case rides an idle H100 node instead; the four A100s keep the 6–0.5 km cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Eliot Quon <eliot@aeolus.earth>
…tecture on_architecture has no Base.ReshapedArray method (unlike SubArray/OffsetArray), so host data arriving reshaped — e.g. a 2-D NetCDF variable reshaped to (Nx, Ny, 1) — fell through the generic identity fallback and reached GPU kernels as CPU memory, failing _set_region_kernel! compilation inside mean_surface_pressure on any GPU nested build. Rewrap the transferred parent instead; TODO upstream the method to Oceananigans.Architectures. Verified: 12 km ERA5→Breeze GPU smoke (previously dying at kernel compilation) now builds and completes 30 iterations with max|w| ≈ 2 m/s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed opts out Adds an `initialize` keyword (default true, preserving the 92dbf96 MOST-coupling density fix for every other caller); nested_atmosphere_model passes initialize = false so the child enters initialize_nested_child! bare, exactly as before 92dbf96 — the construction default should not imprint 285 K / 101325 Pa anchored state on a child whose full state and reference are derived from the parent. Honest scoping: this is NOT a fix for the balance-twin DomainError blowup seen on this branch — that failure reproduces with and without the resting init and is nondeterministic on identical configs (uninitialized-read suspect, tracked toward Breeze#827; evidence table in the run notes). It is a semantic correction that also makes 92dbf96's "nested path unaffected" promise structurally true. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ires The branch tip carries a nondeterministic adiabatic-balance-twin blowup on the CPU path (toward Breeze#827; evidence dossier in scratch_runenv). The sweep pins Breeze to a local branch with only the #821 acoustic-substep reland reverted, and is guarded by the smoke max|w| gate plus a per-run blowup tripwire. Also records the two same-day fixes this campaign surfaced. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Δt=10 instability (Breeze #827) fails differently by architecture: CPU throws a θ^γ DomainError, but GPU's CUDA `pow` returns NaN and the run completes. The prior form keyed the broken marker on "did run! throw" (`stepped`), so on GPU `stepped=true` → @test_broken recorded an unexpected-pass error and the follow-on NaN assertions failed. Fold the full would-be-success check into a single @test_broken block: it records broken on either arch (CPU throw or GPU NaN→false) and flips to an error (prompting re-enable) only once the run integrates cleanly to a finite, changed state again. Verified: CPU test_breeze_coupling is 9 pass / 2 broken / 0 errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…alEarth/NumericalEarth.jl into glw/nested-atmosphere-bc
`time_step!` refreshed the exchanger at the pre-step time `t`. Because the exchanger positions a 2-level window around the passed time, refreshing at `t` leaves the whole step extrapolated whenever `t` lands on a parent time level — i.e. at every hourly ERA5 crossing (12 of them in a 12 h run). Refresh at `t + Δt` so the window brackets the step end and the step's late RK stages stay in-window. A 2-level window still cannot span a parent interval, so on a crossing step a ≤Δt start-side extrapolation remains — acceptable, and far smaller than the prior whole-step extrapolation. `update_state!` is unchanged (no Δt; not the per-step path). Verified: test_nested_simulation StateExchanger window-cycling 8/8 and parent_boundary_conditions 9/9 still pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The child's post-initialization adiabatic (DFI) balance was hardcoded to `set!(nested_model; balancer = true)`. On the ERA5-derived nested IC the default balancer (fully-explicit SSP-RK3 twin at 0.85·Δz_min/c) can drive a pathological cell's pressure negative → Exner DomainError at init, independent of the acoustic-substepper reland (the twin doesn't use it). Thread a `balancer` kwarg (default `true`, preserving behavior) through `nested_atmosphere_model(child_grid, dataset; …)` → `initialize_nested_child!` → the `set!` call. `balancer = false` skips the balance (to isolate whether the interpolated IC steps stably on its own); an `AdiabaticBalancer(Δt=…)` runs a gentler excursion. Enables debugging the init blowup without editing library internals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The derived 2-level window is a pure function of the two resident parent levels, so its values change only when the bracket moves. Recomputing on every intra-interval child step just reproduces identical values, so gate compute_child_prognostics! on `moved || force` and spare the hot path two parent-grid kernels + halo fills per step. `force=true` fills the window once at construction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The nested child blew up at every hourly parent-data seam (t=3600, 7200, ...). Root cause: the StateExchanger held the parent-derived child prognostics in a 2-level windowed FieldTimeSeries, but NestedModel.time_step! brackets t+Δt — so on a step crossing a parent time node the child's start-side sub-stage query (t < node) fell one interval BELOW the resident window and read a stale/wrong boundary + Davies target. In single precision the marginal compressible dynamics amplified that wrong-target kick into a domain-wide NaN at the seam. Fix: use a 3-level window positioned start = clamp(n1-1, 1, N-2) so it spans the crossing and every sub-stage query stays resident. A synthetic 12 h ERA5-nested run now crosses all hourly seams and reaches t=7200 with no relaxation/sponge crutches (base Davies rate, no density ramp, no frame sponge). Also make Interpolated getbc's boundary-evaluation time type-stable by threading the grid float type through clock_time (avoids a Union at Float32/GPU), and drop three stale explicit imports flagged by test_quality_assurance. Adds a deterministic temporal-seam test (windowed crossing-step query is finite, physical, and exact against a linear-in-time parent) and updates the window-cycling test for the 3-level window. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #386. Derives a Breeze nested child's lateral boundary conditions and interior Davies relaxation on the fly from a coarse parent atmosphere's raw state — eliminating the materialized
breeze_prognostic_stateparent series. Builds on theERA5PrescribedAtmosphere/PrescribedAtmosphererefactor from #386 (now onmain).The abstraction
ParentStateBoundary(NestedSimulations) — a multi-field boundary condition: at each boundary-face node it interpolates aNamedTupleof parentFieldTimeSeries(each with its own interpolation transform —logfor the strictly-positivep/T,identityotherwise) and applies a pointwisetransformto the resulting state. Carrier-only; knows nothing about which prognostic or dataset.ParentStateTarget— the interior-relaxation analogue: aRelaxationtarget(x, y, z, t)using the same_query_sourcemachinery at an arbitrary node.interpolate(f, …)(Add interpolate(f, …): map source values through f before blending CliMA/Oceananigans.jl#5726) for the log-space interpolation; the local shim is gone.Breeze wiring (in
NumericalEarthBreezeExt)isbitscallables over the pointwise maps):AirDensity,MomentumDensity,EnergyDensity,MoistureDensity(density-weighted, for the BCs);LiquidIcePotentialTemperature,TotalWater(specific, for the relaxation).atmosphere_simulation(grid; parent_atmosphere, relaxation_rate, relaxation_mask):ρ, ρu, ρv, ρe, <moisture>viaParentStateBoundary(ρu/ρv →NormalFlowBC, the rest →ValueBC), merged per-side with the coupling bottom-flux BCs;(u, v, θ, qᵉ)for Breeze'sSpecificForcing(u/v toward the raw parent velocities, θ/qᵉ toward on-the-fly specific targets).parentkwarg was namedparent_atmosphereto avoid shadowingBase.parent/ Oceananigansparent(field).Workflow: build parent →
atmosphere_simulation(grid; parent_atmosphere)→NestedModel.Dependency
interpolate(f, …)is merged to Oceananigans main but not yet tagged, so this PR points Oceananigans at main via[sources](root/test/docs). Swap to a[compat]bump once a release is tagged (TODOs in place).Status / TODO (draft)
examples/breeze_downscaling_era5.jlto useatmosphere_simulation(grid; parent_atmosphere, relaxation_rate, …), deleting theparent_seriesmaterialization loop + single-sourceparent_boundary_conditions.ParentStateBoundary/ParentStateTarget(CDS-free, toy parent FTS).🤖 Generated with Claude Code