Skip to content

Improve structural lowering and MultiBody simulation parity - #328

Draft
jgoppert wants to merge 2 commits into
mainfrom
msl-trace-parity-50
Draft

Improve structural lowering and MultiBody simulation parity#328
jgoppert wants to merge 2 commits into
mainfrom
msl-trace-parity-50

Conversation

@jgoppert

@jgoppert jgoppert commented Jul 22, 2026

Copy link
Copy Markdown
Member

Branch Naming

  • Descriptive branch name without an agent/ prefix: msl-trace-parity-50

Summary

  • Improves Modelica compilation and simulation across array projection, complex arithmetic, state selection, overconstrained connections, initialization, and solver reconstruction. The verified trace-parity cohort is now 247/566 models (43.6%); this draft does not claim the 50% target has been reached.
  • Adds a general BLT/block-Gaussian condensation pass and cost-aware tearing metadata so dependent accelerations, constraint forces, and causal intermediates can be removed from the nonlinear hot block without changing Modelica equation semantics.
  • Keeps core integrated states in the hot path where dependency analysis permits, while reconstructing cold algebraics for requested outputs, events, projection, and constraint correction.
  • Adds phase and runtime tracing for compilation, initialization, RHS/Jacobian work, projection, roots, row refreshes, and accepted/rejected tearing candidates.
  • Expands MultiBody coverage, including SphericalConstraint, DoublePendulum, DoublePendulumInitTip, and Fourbar2, and updates reusable-booster estimator/variance viewer bindings.
  • Removes the obsolete TODO.md.
  • Addresses SPEC_0007's structural-transformation contracts, SPEC_0022's MLS compliance catalog, and SPEC_0032_DEVELOPMENT_PROCESS's root-cause and oracle-validation requirements.

Fourbar2 results

The root cause was structural rather than the number of integrated states: each RHS call rebuilt a broad algebraic dependency closure and solved a 45-variable coupled block even though the model integrates two states.

  • Exact structural condensation reduces the coupled block from 45 to 21 variables and removes 24 causal variables from the hot nonlinear solve.
  • Derivative refresh rows fall from 114 to 95.
  • Measured RHS time falls from approximately 252.5 ms to 211.9 ms per traced evaluation (about 16%).
  • The structural analysis identifies the same ultimate two-variable core (universalSpherical.f_rod, j1.a) used by OMC's old backend. Forcing the fully inlined 2x2 form was rejected because expression swell increased measured RHS time to approximately 453 ms; the cost model therefore retains the faster exact 21-variable form.
  • At t=0.1, Rumoca and the OMC old-backend oracle agree closely on j1.phi, j1.w, and der(j1.w). OMC's new backend was also checked regularly; it currently stops in NBVariable.makeDummyState for this model, so the old backend remains the numerical oracle.

No model-name conditions, MultiBody-specific rewrites, environment-variable switches, or numerical shortcuts are introduced.

Spec / MLS Alignment

  • Relevant active specs checked: SPEC_0007_IR_PIPELINE, SPEC_0008_PHASE_ERRORS, SPEC_0021_CODE_COMPLEXITY, SPEC_0022_MLS_COMPILER_COMPLIANCE, SPEC_0029_CRATE_BOUNDARIES, SPEC_0032_RANGE_PRESERVING_TENSORS, SPEC_0032_DEVELOPMENT_PROCESS, and SPEC_0025_PR_REVIEW_PROCESS.
  • Relevant MLS sections: §4.4.2.2 (component prefixes), §4.9.5 (enumeration ordinals), §8 and Appendix B (equation/DAE semantics and initialization), §9.4 (overconstrained connection graphs), §10.4.2 and §10.6 (array construction and array reduction/vector semantics).
  • Crate/phase owners: flattening and DAE lowering own MLS equation formation; rumoca-phase-structural owns matching, state selection, BLT, exact elimination, and tearing metadata; rumoca-phase-solve owns solve IR; eval/sim/solver crates own dependency-scoped reconstruction and runtime evaluation.

Risk and Design Notes

  • Main correctness risk: symbolic elimination or cold reconstruction could change equation ownership, array index projection, event dependencies, or initialization behavior. The pass only accepts exact one-to-one causal pivots, preserves aggregate ownership and equation/variable counts, and falls back without transformation when the solver view is unsuitable.
  • Main maintenance risk: the fix necessarily crosses several established phase boundaries. Shared primitives and metadata live in their owning IR/structural crates, while runtime execution stays in eval/sim/solver modules; no duplicate old/new semantic paths are retained.
  • Why the change belongs in these crates: compile-time structural analysis decides what is mathematically eliminable; solve lowering serializes that decision; runtime code executes only the resulting dependency plan.
  • New abstractions/public API: reusable causal-factor and block-condensation analysis, dummy-derivative grouping, causal reconstruction metadata, and cost-aware runtime tearing. Existing models and callers require no migration.

Testing

  • Key commands run:
    • nix develop -c cargo fmt --all --check
    • cargo test -p rumoca-phase-structural — 301 passed
    • cargo test -p rumoca-sim — 45 passed
    • nix develop -c cargo clippy -p rumoca-phase-structural -p rumoca-sim --all-targets --all-features -- -D warnings
    • nix develop -c cargo xtask verify lint — workspace lint verification passed
    • git diff --check origin/main...HEAD
  • Behavior/regressions covered: exact block condensation and safe fallback, causal factorization, dummy derivative/state selection, array/scalar projection, complex field arithmetic, reverse residual sensitivities, dependency-scoped refresh/output reconstruction, solver projection/tearing, and the MultiBody examples listed above.
  • Traced Fourbar2 against both OMC backends; compared old-backend numerical outputs and inspected new-backend failure location.
  • Commands not run locally: full cargo test --workspace, cargo doc --no-deps, the full MSL gate, ModelicaTest semantic gate, and pinned modelica_models gate. This remains a draft until those heavyweight repository/CI gates complete and parity artifacts are reviewed.
  • MSL gate: focused parity and MultiBody runs were used during development; the promotable full gate was not run locally, and no partial snapshot is promoted.

Code Size Budget (required)

  • production_lines_added: 9160
  • production_lines_deleted: 1010
  • test_lines_added: 2414
  • test_lines_deleted: 25
  • public_items_added: 37
  • public_items_removed: 2
  • files_touched: 103
  • net_added_lines: 10539

Public-item counts use a mechanical diff of externally visible Rust declarations; line counts use git diff --numstat origin/main...HEAD.

  • Why this net growth is required: the change closes multiple independently reproduced MLS/array/state-selection failures, adds the general structural transformation and runtime dependency plan, and adds regression coverage across every affected phase rather than encoding model-specific exceptions.
  • First compression pass: expression-identity logic, runtime-plan code, projection code, and obsolete scaffolding were consolidated or removed (1,035 total deleted lines); large algorithms were extracted into phase-owned modules to keep review and complexity boundaries explicit.
  • Follow-up cleanup before marking ready: consolidate repeated state-selection fixtures and profiling inventory helpers after the full parity ratchet stabilizes; keep the draft open until that compression opportunity and the heavyweight gates are reviewed.

Reviewer Checklist

  • Relevant active specs were checked.
  • MLS-sensitive changes cite the relevant MLS sections.
  • Crate boundaries and phase ownership are preserved.
  • Focused tests prove the changed behavior.
  • Standard CI gates pass in full (cargo test --workspace and cargo doc remain).
  • Full MSL, ModelicaTest, and pinned-corpus gates complete without regression.
  • Size-budget section is complete with positive-growth justification.
  • New APIs are phase-owned and exercised.
  • No parallel old/new semantic paths were added.
  • No #[allow(clippy::...)] was added outside generated code.
  • Every commit is signed off and contains no co-author trailer.
  • No external source material was copied.

jgoppert added 2 commits July 22, 2026 11:28
@jgoppert
Improve Modelica structural lowering and simulation parity
89bb758 
Signed-off-by: James Goppert <james.goppert@gmail.com>
@jgoppert
Update reusable booster scenario bindings
41e324d 
Signed-off-by: James Goppert <james.goppert@gmail.com>
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

MSL Quality Gate Summary

Generated by cargo xtask repo msl pr-comment from CI result artifacts.

Scope MSL OMC Compile Balance Initial Balance Simulation
full v4.1.0 a96aa1a-cmake 541/528 (Δ-4/-4) 528/528 (Δ-4/-4) 528/528 (Δ-4/-4) 204/566 (Δ+34/0)

Deltas compare numerator/denominator against the resolved MSL quality baseline.
Baseline: quality gate v1, commit 08fac54, MSL v4.1.0, OMC OpenModelica 1.27.0~dev.beta.3.
Trace agreement vs baseline: high+near Δ+21, deviation Δ+13.

CI Gate Snapshot

These are the baseline-relative MSL stats checked by CI; IC progress is contextual unless simulation successes regress, and speed metrics are informational only.

Gate area Current (Δ vs baseline)
Structural floors parse 566 (0), flat 565 (0), DAE 541 (-4), solve 451 (+70)
Balance floors balance 528/528 (-4), init 528/528 (-4)
Runtime gate IC progress 282/566 (+43), sim 204/566 (+34)
Trace floors acceptable 171/566 (+21), no-severe 160/566 (+11), compared 200 (+34)
CI gate details
CI gate metric Current Baseline Δ
Parse floor 566/566 566/566 0
Flatten floor 565/566 565/566 0
DAE/compile floor 541/566 545/566 -4
Solve-IR floor 451/566 381/566 +70
Balance floor 528/528 532/532 -4
Initial balance floor 528/528 532/532 -4
IC progress 282/566 239/566 +43
Simulation floor 204/566 170/566 +34
Partial models ceiling 13 13 0
Unbalanced models ceiling 0 0 0
Trace acceptable floor 171/566 150/566 +21
Trace no-severe floor 160/566 149/566 +11
Trace compared floor (drop ≤2) 200 166 +34

Package Pass Rates

MSL Package n Ast Flat Dae Solve IC Sim
Overall 566 100% 100% 96% 81% 44% 30%
Per-package pass rates
MSL Package n Ast Flat Dae Solve IC Sim
Blocks 32 100% 100% 97% 84% 47% 38%
Clocked 76 100% 100% 100% 100% 83% 80%
ComplexBlocks 2 100% 100% 100% 100% 50% 50%
Electrical.Analog 65 100% 100% 100% 88% 58% 31%
Electrical.Batteries 8 100% 100% 50% 50% 13% 0%
Electrical.Digital 23 100% 100% 100% 100% 96% 91%
Electrical.Machines 43 100% 100% 100% 98% 21% 5%
Electrical.Polyphase 5 100% 100% 100% 80% 80% 0%
Electrical.PowerConverters 59 100% 100% 100% 76% 49% 22%
Electrical.QuasiStatic.Machines 1 100% 100% 100% 0% 0% 0%
Electrical.QuasiStatic.Polyphase 4 100% 100% 100% 100% 0% 0%
Electrical.QuasiStatic.SinglePhase 6 100% 100% 100% 100% 0% 0%
Electrical.Spice3 14 100% 100% 93% 79% 14% 7%
Fluid 23 100% 100% 57% 4% 0% 0%
Magnetic.FluxTubes 20 100% 100% 100% 95% 10% 5%
Magnetic.FundamentalWave 27 100% 100% 100% 67% 7% 0%
Magnetic.QuasiStatic.FluxTubes 9 100% 100% 100% 100% 22% 0%
Magnetic.QuasiStatic.FundamentalWave 21 100% 100% 76% 19% 0% 0%
Math.FastFourierTransform 2 100% 100% 100% 100% 0% 0%
Math.Nonlinear 1 100% 100% 100% 100% 100% 100%
Math.Random 1 100% 100% 100% 100% 0% 0%
Mechanics.MultiBody 42 100% 100% 100% 88% 31% 31%
Mechanics.Rotational 17 100% 100% 100% 100% 76% 41%
Mechanics.Translational 16 100% 100% 100% 100% 75% 44%
Media 22 100% 100% 100% 59% 0% 0%
Media.Incompressible 1 100% 100% 100% 0% 0% 0%
StateGraph 7 100% 100% 57% 57% 57% 57%
Thermal.FluidHeatFlow 12 100% 100% 100% 100% 75% 25%
Thermal.HeatTransfer 4 100% 100% 100% 100% 75% 75%
Utilities 3 100% 67% 67% 67% 67% 33%
MLS Contract Coverage
MLS Category n Compile Solve IR Balance Sim Phases Errors
ARR 9 89% 89% 89% 89% Flatten:1, Success:8 -
CLK_SM 77 100% 96% 96% 84% Success:74, ToDae:3 -
CONN_STRM 339 100% 80% 96% 31% Success:324, ToDae:15 -
EQN_ALG_SIM 2 100% 50% 50% 0% Success:1, ToDae:1 -
FUNC 1 100% 100% 100% 0% Success:1 -
OTHER 138 100% 70% 96% 18% Success:133, ToDae:5 -
Trace Accuracy vs OMC
MSL Package n Compared Trace High Near Channel OK No Severe
Blocks 32 44% 38% 34% 3% 84% 41%
Clocked 76 86% 80% 74% 7% 90% 84%
ComplexBlocks 2 50% 50% 50% 0% 100% 50%
Electrical.Analog 65 37% 31% 17% 14% 92% 26%
Electrical.Batteries 8 0% 0% 0% 0% 0% 0%
Electrical.Digital 23 96% 91% 91% 0% 85% 96%
Electrical.Machines 43 5% 5% 2% 2% 97% 5%
Electrical.Polyphase 5 20% 0% 0% 0% 66% 0%
Electrical.PowerConverters 59 25% 22% 0% 22% 92% 3%
Electrical.QuasiStatic.Machines 1 0% 0% 0% 0% 0% 0%
Electrical.QuasiStatic.Polyphase 4 0% 0% 0% 0% 0% 0%
Electrical.QuasiStatic.SinglePhase 6 67% 0% 0% 0% 72% 0%
Electrical.Spice3 14 7% 7% 7% 0% 100% 7%
Fluid 23 0% 0% 0% 0% 0% 0%
Magnetic.FluxTubes 20 5% 5% 0% 5% 97% 0%
Magnetic.FundamentalWave 27 0% 0% 0% 0% 0% 0%
Magnetic.QuasiStatic.FluxTubes 9 56% 0% 0% 0% 63% 0%
Magnetic.QuasiStatic.FundamentalWave 21 0% 0% 0% 0% 0% 0%
Math.FastFourierTransform 2 0% 0% 0% 0% 0% 0%
Math.Nonlinear 1 100% 100% 100% 0% 100% 100%
Math.Random 1 0% 0% 0% 0% 0% 0%
Mechanics.MultiBody 42 38% 31% 7% 24% 95% 24%
Mechanics.Rotational 17 47% 41% 41% 0% 96% 41%
Mechanics.Translational 16 56% 44% 31% 13% 95% 56%
Media 22 0% 0% 0% 0% 0% 0%
Media.Incompressible 1 0% 0% 0% 0% 0% 0%
StateGraph 7 57% 57% 43% 14% 100% 57%
Thermal.FluidHeatFlow 12 25% 25% 25% 0% 100% 25%
Thermal.HeatTransfer 4 75% 75% 50% 25% 99% 75%
Utilities 3 33% 33% 33% 0% 100% 33%
Overall 566 35% 30% 22% 8% 90% 28%

Speed vs OMC

Compilation and simulation are reported separately for the 168 trace-agreeing models with valid timings on both tools. Speedup = OMC / Rumoca (>1 means rumoca faster).

Aggregate speed summary
Phase Models Rumoca total (s) OMC total (s) Throughput speedup (×) Median per-model speedup (×)
Total 168 423.4 675.9 1.60 5.61
Compilation 168 186.9 625.6 3.35 5.87
Simulation 168 236.5 50.3 0.21 3.28
Speed breakdown
Where rumoca's time goes
Scalar eqns Models front→DAE Solve-IR + JIT integration
1–9 28 0.504 0.009 0.0011
10–24 30 0.531 0.012 0.0024
25–49 25 0.544 0.018 0.0054
50–99 37 0.534 0.077 0.0226
100–249 26 0.714 0.211 0.0629
250+ 22 0.982 2.023 0.3462
Rumoca stage Total seconds Share of rumoca total
front→DAE 114.5 27%
Solve-IR + JIT 72.4 17%
integration 236.5 56%
Speed by system size
Table 1 — Total (model → simulated results)

Rumoca = front-end compile + Solve-IR/JIT build + integration; OMC = timeTotal.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.5158 3.2943 6.30
10–24 30 0.5459 3.4318 6.29
25–49 25 0.5984 3.5730 5.92
50–99 37 0.6372 3.8202 5.62
100–249 26 1.5902 4.6829 2.77
250+ 22 4.2821 6.7463 1.53

Per-model speedup: min 0.08, median 5.61, max 9.67.

Table 2 — Compilation (build-to-runnable)

Rumoca = front-end compile + Solve-IR/JIT build; OMC = timeTotal - timeSimulation.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.5148 3.2798 6.28
10–24 30 0.5432 3.4144 6.29
25–49 25 0.5788 3.5348 5.96
50–99 37 0.5679 3.6051 5.88
100–249 26 1.1424 4.5215 3.79
250+ 22 3.4704 6.1673 1.81

Per-model speedup: min 0.58, median 5.87, max 9.66.

Table 3 — Simulation (integration only)

Rumoca = sim_run_seconds; OMC = timeSimulation.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.0011 0.0145 13.38
10–24 30 0.0024 0.0164 6.73
25–49 25 0.0054 0.0194 3.94
50–99 37 0.0226 0.0504 2.59
100–249 26 0.0629 0.0643 0.77
250+ 22 0.3462 0.1347 0.40

Per-model speedup: min 0.00, median 3.28, max 121.26.

Top 10 slowest models
Top 10 slowest rumoca compilation models

Trace-agreeing models ranked by rumoca time.

Model Scalar eqns Rumoca compile (s) OMC compile (s) Speedup (×)
Modelica.Mechanics.MultiBody.Examples.Loops.Fourbar2 2242 11.5122 6.6410 0.58
Modelica.Mechanics.MultiBody.Examples.Elementary.DoublePendulumInitTip 1479 10.5035 9.8077 0.93
Modelica.Mechanics.MultiBody.Examples.Elementary.PendulumWithSpringDamper 1349 6.7504 8.3674 1.24
Modelica.Mechanics.MultiBody.Examples.Constraints.SphericalConstraint 2320 5.8883 6.5204 1.11
Modelica.Electrical.Machines.Examples.DCMachines.DCPM_withLosses 399 4.5237 5.8142 1.29
Modelica.Mechanics.MultiBody.Examples.Elementary.SpringMassSystem 1469 4.4596 7.5784 1.70
Modelica.Mechanics.MultiBody.Examples.Elementary.DoublePendulum 1302 4.3193 4.9647 1.15
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_RL 498 3.8414 3.5325 0.92
Modelica.Mechanics.MultiBody.Examples.Elementary.SpringWithMass 968 3.8156 7.9034 2.07
Modelica.Mechanics.MultiBody.Examples.Rotational3DEffects.GearConstraint 1638 3.7680 5.2522 1.39
Top 10 slowest rumoca simulation models

Trace-agreeing models ranked by rumoca time.

Model Scalar eqns Rumoca simulation (s) OMC simulation (s) Speedup (×)
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_RL 498 40.9710 0.1661 0.00
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.ThyristorBridge2mPulse_RLV 503 40.7069 0.1704 0.00
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTapmPulse.ThyristorCenterTapmPulse_R 367 27.1239 0.0974 0.00
Modelica.Electrical.Machines.Examples.DCMachines.DCPM_withLosses 399 24.1470 5.6697 0.23
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2Pulse.ThyristorBridge2Pulse_RL 190 13.8220 0.1224 0.01
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTapmPulse.ThyristorCenterTapmPulse_RL 373 13.7487 0.1001 0.01
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2Pulse.ThyristorBridge2Pulse_RLV 196 12.6253 0.1201 0.01
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2Pulse.ThyristorCenterTap2Pulse_RLV 176 8.7504 0.1036 0.01
Modelica.Mechanics.Rotational.Examples.OneWayClutch 58 7.8784 0.0925 0.01
Modelica.Mechanics.Rotational.Examples.OneWayClutchDisengaged 56 6.6285 0.0891 0.01

Additional Artifacts

  • msl_compatibility_report.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jgoppert