Skip to content

feat(domain): add 2D solver-in-the-loop corrector benchmark#116

Draft
andrinr wants to merge 15 commits into
mainfrom
feat/ns-grid-solver-in-loop
Draft

feat(domain): add 2D solver-in-the-loop corrector benchmark#116
andrinr wants to merge 15 commits into
mainfrom
feat/ns-grid-solver-in-loop

Conversation

@andrinr

@andrinr andrinr commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add a solver-in-the-loop neural-correction benchmark to 2D Navier–Stokes
alongside the existing drag-optimization task.

The benchmark owns the train/test split, references, Equinox corrector,
initialization, optimizer, recurrent objective, paired stop-gradient control,
and evaluation. Candidate cells vary only in the differentiable solver
transition and its VJP.

The central result is a two-stage distinction:

  1. all six differentiable solvers pass the same analytic forward control;
  2. only PICT and Warp-NS currently preserve enough state across the canonical
    Tesseract boundary to pass a 100-call recurrence test.

No solver is hidden to make the comparison agree. The four failed cells remain
visible as adapter diagnostics, and the complete intrinsic six-solver VJP
ranking is withheld until their recurrent state is preserved.

Type of change

  • New solver backend
  • Solver tuning / improvement
  • New benchmark domain
  • Harness / infrastructure change
  • Documentation
  • Bug fix

Corrector and paired training

The corrector is an Equinox PeriodicResidualCNN, not a U-Net:

  • circularly padded 5×5 convolutions, channels 2 → 32 → 64 → 2;
  • 56,098 identical parameters for every solver;
  • zero-initialized residual head, so update zero is exactly solver-only;
  • additive correction after each solver interval;
  • zero-mean, divergence-free spectral projection of each correction.

Every model seed is trained twice:

  • full VJP: temporal credit propagates through the differentiable solver;
  • stop-gradient: the identical forward solver recurrence and local
    supervision are retained, but temporal credit through the solver is cut.

The reported solver-VJP lift is the paired
log(error_stop-gradient / error_full-VJP) over the same seeds and held-out
ICs. Each cell's loss is normalized by its own uncorrected recurrent baseline,
so raw solver accuracy does not change the optimizer scale.

Fair all-six forward gate

The analytic control is the smooth N=64 Taylor–Green case from the established
forward benchmark (nu=0.05, dt=0.01). Admission thresholds are declared as
1% relative error for the first interval, 1% for an uninterrupted t=1
trajectory, and 1% for repeated/native closure after 100 canonical calls.

XLB receives four internal substeps per canonical interval. This is the
smallest tested Mach-safe budget that makes its uninterrupted forward solution
pass the same 1% gate; two substeps left 1.79% long-forward error.

Solver First interval Uninterrupted t=1 100-call closure Ranking eligible
JAX-CFD 0.243% 0.441% 21.203% no
INS.jl 0.241% 0.233% 21.230% no
PhiFlow 0.241% 0.238% 21.230% no
PICT 0.0056% 0.0122% 0.000% yes
Warp-NS 0.0090% 0.0098% 0.000% yes
XLB 0.677% 0.804% 10.351% no

This resolves the apparent contradiction in the earlier plots: the native
forward trajectories are accurate, while repeated velocity-only calls are not
the same state transition for four adapters.

  • JAX-CFD, INS.jl, and PhiFlow discard native staggered face velocity. Repeated
    center↔face conversions attenuate the state.
  • XLB discards density and non-equilibrium population moments, then rebuilds
    equilibrium populations from velocity on every call.
  • PICT and Warp-NS have velocity-complete periodic transitions and close at
    numerical precision.

These are interface-state failures, not evidence that the underlying native
solver VJPs are intrinsically poor.

All-solver forward calibration

All-solver forward and recurrence admission

Nonlinear shared-reference task

The learning stress task uses a benchmark-owned 64² dealiased pseudo-spectral
vorticity RK4 reference, spectrally restricted to the shared 32² grid. It is
not an upscaled candidate solver.

  • multimode k0=6, amplitude 0.5, nu=0.001;
  • four native steps of dt=0.02 per correction interval;
  • 16 training ICs (seeds 0–15) and eight disjoint held-out ICs (100–107);
  • three paired model seeds and 200 optimizer updates;
  • recurrent training to t=1.92, held-out rollout to t=2.88.

The flow changes materially over the rollout and uses doubly periodic
[0,2π]² boundaries, appropriate for homogeneous decaying turbulence and the
translation-equivariant corrector. The benchmark does not claim wall or inlet
generalization; the retained drag task covers that separate regime.

All six cells run and render. PICT and Warp-NS are the conditional intrinsic
comparison because they pass recurrence admission and begin at matched
first-interval p95 error (6.32% and 6.28%).

Quantity PICT Warp-NS
Geometric correction gain 2.091× 1.608×
Final held-out error 0.182 0.207
Incremental solver-VJP lift 1.08% 3.34%
Paired bootstrap 95% CI −0.69% to 2.40% 2.09% to 4.44%
Median full-VJP update 2.234 s 0.846 s

Warp-NS's incremental VJP lift is distinguishably larger: its lift ratio over
PICT is 1.022× (95% CI 1.008–1.036×). PICT nevertheless learns the stronger
overall correction and lower final error. Warp-NS is about 2.6× faster per
full-VJP update.

So the differentiable solver VJP does help, but modestly. In this nonlinear
task the evidence favors Warp-NS for temporal credit assignment and efficiency,
while PICT is more correctable in absolute terms.

Nonlinear held-out trajectory

Reference, solver-only, and corrected nonlinear fields

Conditional nonlinear comparison

Solver-specific refined-reference control

optimization/solver_in_loop_self_reference asks the upscaling question:

For a recurrence-admitted solver, does differentiating through its coarse
transition help imitate that solver's own higher-resolution, smaller-step
trajectory?

Each solver supplies its own 64², half-step reference, restricted to the 32²
coarse grid. Absolute errors are not compared across solvers because their
targets differ; the paired within-solver VJP lift is comparable.

Quantity PICT Warp-NS
Geometric correction gain 1.923× 1.527×
Incremental solver-VJP lift 3.56% 4.05%
Paired bootstrap 95% CI 1.72% to 5.27% 2.32% to 5.83%
Median full-VJP update 2.214 s 0.810 s

Both VJPs help at 95% confidence, but their lift difference is not significant.
Warp-NS remains about 2.7× faster.

Solver-specific reference fields

Solver-specific paired comparison

Solver-specific held-out trajectory

Analytic TGV null control

The all-six TGV experiment is intentionally smooth and near the error floor.
Among the admitted adapters, PICT and Warp-NS obtain incremental VJP lifts of
2.19% and 0.69%; total correction gains are 1.421× and 1.055×. This confirms
the paired machinery but is not a substitute for the nonlinear learning task.

TGV reference, solver-only, and corrected fields

TGV conditional comparison

TGV held-out trajectory

Supporting JSON, complete plot inputs, physics diagnostics, and job provenance:
local artifact index.

Other fixes included

  • Select the reference per forward-agreement run. Multimode fields now use
    cross-solver consensus instead of being compared with an unrelated analytic
    TGV field.
  • Repair the PICT projection path and build its runnable container through the
    daemon-free Slurm/Pyxis workflow.
  • Preserve the existing drag-optimization registrations.
  • Keep all Slurm/build helper tooling outside this repository.

Validation

All experiments and validation ran through Slurm on Kander.

  • nonlinear production: JAX-CFD 1642897, INS.jl 1642898,
    PhiFlow 1642899, PICT 1642900, Warp-NS 1642901, XLB 1642902;
    merge/render 1644051;
  • self-reference production: PICT 1642930, Warp-NS 1642931;
    merge/render 1644052;
  • final all-six TGV: JAX-CFD 1644252, INS.jl 1644253,
    PhiFlow 1644254, PICT 1644255, Warp-NS 1644256, XLB 1644258;
    merge/render 1644264;
  • final Ruff 1644419, format 1644422, full pytest 1644423
    (499 passed, three skipped), problem configs 1644424, Tesseract configs
    1644425, git-aware pre-commit 1644426, artifact integrity 1644825.

Generated benchmark results live only on the artifact branch. This PR remains
a draft with no benchmark label while the comparison and native-state follow-up
are reviewed.

@andrinr
andrinr force-pushed the feat/ns-grid-solver-in-loop branch from 2b53888 to 358e231 Compare July 24, 2026 15:02
@andrinr andrinr added benchmark:solver Benchmark only the modified solver and removed benchmark:solver Benchmark only the modified solver labels Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant