Skip to content

fix(msl): add tensor-aware compiler and solver support for MultiBody - #325

Merged
jgoppert merged 1 commit into
mainfrom
msl-trace-parity-recovery
Jul 21, 2026
Merged

fix(msl): add tensor-aware compiler and solver support for MultiBody#325
jgoppert merged 1 commit into
mainfrom
msl-trace-parity-recovery

Conversation

@jgoppert

@jgoppert jgoppert commented Jul 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Branch Naming

  • msl-trace-parity-recovery is descriptive and has no agent/ prefix.

Summary

  • Adds the compiler semantics needed by MSL MultiBody rather than a model-specific workaround: preserves each modifier origin and record bindings, resolves record/function projections exactly, recovers array dimensions, and removes suffix-based lookup ambiguity through instantiation, flattening, and DAE conversion.
  • Generalizes structural processing for MultiBody systems: derivative closure and symbolic function/record derivatives, exact stateSelect metadata, dummy-state demotion, vector/tensor constraint row shapes, index reduction, preferred unknown matching, and regular-subsystem BLT construction.
  • Keeps compact tensor/range-native IR authoritative. Structural rewrites operate before the solver scalar view; indexed field identity, matrix/vector products, transpose, slices, ranges, comprehensions, constructors, cross/skew/outer products, and projected record outputs are scalarized only at the owning boundary.
  • Extends Solve IR and runtime with tensor-native JVPs, explicit initialization/algebraic projection plans, exact SolverY versus [Y|P] sensitivity domains, prepared assignment shapes/output ownership, and a complete causal certificate. The simultaneous residual solve remains authoritative whenever that certificate is incomplete.
  • Corrects FMI 3 event and zero-domain behavior, including relation memory, event iteration, scheduled events, zero-sized nested arrays, co-simulation, and generated runtime coverage.
  • Pins the authoritative OpenModelica CLI through flake.nix and CogniPilot Cachix, removes the Debian installation route, and adds an explicit fail-closed OMC baseline-context migration for the completed 566-model run.
  • Reduces the reusable-booster warm direct route from 629.56 s to 43.42 s (14.5x) without weakening the residual solver.
  • This PR is the complete MultiBody-enabling compiler/runtime change, not only a history consolidation. It does not claim completion of the separate 283/566 parity goal.

Spec / MLS Alignment

  • Active specs checked: SPEC_0007, SPEC_0021, SPEC_0022, SPEC_0025, SPEC_0029, SPEC_0032 development process, and SPEC_0032 range-preserving tensors.
  • MLS areas checked: §7.2.5 each modifiers; §8.6 initialization and simultaneous equations; §10 array construction, operators, slicing, and subscripts; §12.4.1 named/default function arguments and structured outputs; Appendix B DAE semantics.
  • Phase ownership remains explicit: instantiate owns modifier origin; flatten/DAE own exact record and function identity; structural owns state selection, matching, and index reduction; solve owns scalar views/JVP/projection artifacts; solver/codegen consume those artifacts.
  • No model-name branches, inferred-length heuristic, legacy/backward-compatibility path, dense tensor replacement, or speculative semantic fallback is retained.

Risk and Design Notes

  • Main correctness risk: generalized derivative/state reduction can change matching and physical state selection. Exact alias metadata is propagated, StateSelect.always is preserved, and focused vector-constraint/dummy-state regressions cover the new behavior.
  • Runtime fast-path risk is bounded by proof obligations: complete non-state output inventory, unique exact assignments, singleton BLT ownership, valid dependencies, and a complete DAG. Failure of any obligation uses the simultaneous residual path.
  • The checked-in OMC migration must exactly match promoted source version, checked-in target version, and the fixed 566 target count. Missing, malformed, reversed, stale, or internally inconsistent declarations fail; downstream artifacts must still match the selected OMC context.
  • Maintenance risk is the cross-phase breadth. Shared logic was moved into phase-owned helpers and cohesive modules to satisfy code-size limits; old and new implementations were not left side by side.
  • The adversarial reviewer performed static review only, ran no commands, and wrote ignored reports under dev/reviews/. Its final review found no blocker or high-severity issue.

Testing

Completed on the squashed tree or its identical pre-squash tree:

  • nix develop -c cargo fmt --all -- --check: pass.
  • nix develop -c cargo clippy --workspace --all-targets --all-features -- -D warnings: pass.
  • nix develop -c cargo xtask verify workspace: pass.
  • nix develop -c cargo doc --workspace --all-features --no-deps: pass.
  • nix develop -c cargo xtask verify examples: 42/42, including CMM and reusable-booster.
  • Eval-solve: 128/128; structural: 275/275; phase-solve: 834/834; flatten: 415/415.
  • Diffsol: 59/59; RK45: 23/23.
  • FMI 3 runtime: 27/27; FMI 3 codegen: 17/17; FMI metadata: 15/15.
  • Focused MultiBody cohort: 10/10 compile, 10/10 balance, 8/10 simulate; 5 high, 1 near, 2 deviations against OMC. The same two known structural singularities remain.
  • Baseline selector: 7/7 focused tests covering exact/equal/undeclared/reversed/offline/malformed context behavior.
  • Exact four-shard merge/gate rerun on the squashed code: pass.
  • Exact pushed-SHA CI #1406: 36 required jobs passed, 4 conditional deploy/release jobs skipped, zero failures. This includes Windows/macOS/Linux, lint/architecture, coverage, docs, CMM/examples, FMI/template runtimes, WASM, Python, VSCode/editor, ModelicaTest, modelica_models, all four MSL shards, and the merged quality gate: https://github.com/CogniPilot/rumoca/actions/runs/29831013519

Fresh fixed-denominator MSL evidence from CI #1406:

  • Parse 566/566; flatten 564/566; DAE 544/566; Solve IR 443/566.
  • Structurally balanced 531/531; initial balance 531/531; initial conditions 252/566; simulated 207/566.
  • OMC compared 202; high 143; near 45; deviation 14.
  • Acceptable OMC trace parity is 188/566. This headline count is 38 above the promoted-main 150/566 snapshot, while the strict gate uses the selected matching OMC context rather than comparing regression metrics across reference-compiler versions.
  • No-severe traces are 164/566 and 185 trace-agreeing models have valid timing on both tools.
  • Hosted-runner stage jitter was within the full-gate policy tolerance; simulation, comparison, and acceptable-trace counts matched the checked-in full snapshot exactly.
  • Full MultiBody package: 42/42 DAE, 81% Solve IR, 6/42 acceptable OMC traces.
  • OMC context: flake-pinned a96aa1a-cmake.
  • The result remains 95 models short of the 283/566 goal.

Code Size Budget

  • production_lines_added: 9325
  • production_lines_deleted: 7222
  • test_lines_added: 9773
  • test_lines_deleted: 3904
  • public_items_added: 136
  • public_items_removed: 39
  • files_touched: 156
  • net_added_lines: 7972

Positive growth is concentrated in generalized structural/tensor lowering, explicit solve/runtime contracts, FMI 3 behavior, and regression coverage. The diff also deletes 11,126 lines and splits oversized modules without retaining duplicate implementations. Further compression should follow measured MSL cluster work and preserve phase-owned metadata and focused tests.

Reviewer Checklist

  • Relevant active specs and MLS sections checked.
  • Compiler, IR, runtime, and codegen ownership identified.
  • Tensor/range-native structures preserved.
  • MultiBody gains and remaining singularities disclosed.
  • Fresh fixed-566 MSL evidence reported without claiming 283/566 completion.
  • CMM, FMI 3, ModelicaTest, focused regressions, and full MSL shards retained.
  • No legacy path, model-specific branch, or speculative fallback retained.
  • Static adversarial review completed with no blocker/high finding.
  • One signed-off commit directly atop main; no AI co-author trailer.
  • Size budget and positive-growth justification included.
  • No external source material copied.
  • Exact pushed-SHA CI passes.
  • At least one approving external review and no unresolved conversations.

@jgoppert jgoppert changed the title fix(msl): recover trace parity while retaining MultiBody fix(msl): consolidate parity, MultiBody, FMI 3, and runtime performance Jul 21, 2026
@jgoppert
jgoppert marked this pull request as ready for review July 21, 2026 11:46
@github-actions

github-actions Bot commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown

MSL Quality Gate Summary

Generated by cargo xtask repo msl pr-comment from CI result artifacts.

Scope MSL OMC Compile Balance Initial Balance Simulation
full v4.1.0 a96aa1a-cmake 544/531 (Δ-1/-1) 531/531 (Δ-1/-1) 531/531 (Δ-1/-1) 207/566 (Δ+37/0)

Deltas compare numerator/denominator against the resolved MSL quality baseline.
Baseline: quality gate v1, commit 08fac54, MSL v4.1.0, OMC OpenModelica 1.27.0~dev.beta.3.
Trace agreement vs baseline: high+near Δ+38, deviation Δ-2.

CI Gate Snapshot

These are the baseline-relative MSL stats checked by CI; IC progress is contextual unless simulation successes regress, and speed metrics are informational only.

Gate area Current (Δ vs baseline)
Structural floors parse 566 (0), flat 564 (-1), DAE 544 (-1), solve 443 (+62)
Balance floors balance 531/531 (-1), init 531/531 (-1)
Runtime gate IC progress 252/566 (+13), sim 207/566 (+37)
Trace floors acceptable 188/566 (+38), no-severe 164/566 (+15), compared 202 (+36)
CI gate details
CI gate metric Current Baseline Δ
Parse floor 566/566 566/566 0
Flatten floor 564/566 565/566 -1
DAE/compile floor 544/566 545/566 -1
Solve-IR floor 443/566 381/566 +62
Balance floor 531/531 532/532 -1
Initial balance floor 531/531 532/532 -1
IC progress 252/566 239/566 +13
Simulation floor 207/566 170/566 +37
Partial models ceiling 13 13 0
Unbalanced models ceiling 0 0 0
Trace acceptable floor 188/566 150/566 +38
Trace no-severe floor 164/566 149/566 +15
Trace compared floor (drop ≤2) 202 166 +36

Package Pass Rates

MSL Package n Ast Flat Dae Solve IC Sim
Overall 566 100% 100% 96% 81% 39% 33%
Per-package pass rates
MSL Package n Ast Flat Dae Solve IC Sim
Blocks 32 100% 100% 97% 84% 41% 47%
Clocked 76 100% 100% 100% 100% 82% 80%
ComplexBlocks 2 100% 100% 100% 100% 50% 50%
Electrical.Analog 65 100% 100% 100% 88% 57% 37%
Electrical.Batteries 8 100% 100% 50% 50% 13% 13%
Electrical.Digital 23 100% 100% 100% 100% 96% 91%
Electrical.Machines 43 100% 100% 100% 98% 0% 0%
Electrical.Polyphase 5 100% 100% 100% 80% 60% 40%
Electrical.PowerConverters 59 100% 100% 100% 76% 37% 44%
Electrical.QuasiStatic.Machines 1 100% 100% 100% 100% 0% 0%
Electrical.QuasiStatic.Polyphase 4 100% 100% 100% 100% 0% 0%
Electrical.QuasiStatic.SinglePhase 6 100% 100% 100% 100% 0% 0%
Electrical.Spice3 14 100% 93% 86% 79% 7% 7%
Fluid 23 100% 100% 57% 4% 0% 0%
Magnetic.FluxTubes 20 100% 100% 100% 95% 15% 0%
Magnetic.FundamentalWave 27 100% 100% 100% 4% 0% 0%
Magnetic.QuasiStatic.FluxTubes 9 100% 100% 100% 100% 0% 0%
Magnetic.QuasiStatic.FundamentalWave 21 100% 100% 95% 86% 0% 0%
Math.FastFourierTransform 2 100% 100% 100% 100% 0% 0%
Math.Nonlinear 1 100% 100% 100% 100% 100% 100%
Math.Random 1 100% 100% 100% 100% 0% 0%
Mechanics.MultiBody 42 100% 100% 100% 81% 14% 14%
Mechanics.Rotational 17 100% 100% 100% 100% 88% 47%
Mechanics.Translational 16 100% 100% 100% 100% 81% 63%
Media 22 100% 100% 100% 59% 5% 0%
Media.Incompressible 1 100% 100% 100% 0% 0% 0%
StateGraph 7 100% 100% 57% 57% 57% 57%
Thermal.FluidHeatFlow 12 100% 100% 100% 100% 75% 25%
Thermal.HeatTransfer 4 100% 100% 100% 100% 75% 75%
Utilities 3 100% 67% 67% 67% 67% 33%
MLS Contract Coverage
MLS Category n Compile Solve IR Balance Sim Phases Errors
ARR 9 89% 89% 89% 89% Flatten:1, Success:8 -
CLK_SM 77 100% 96% 96% 84% Success:74, ToDae:3 -
CONN_STRM 339 100% 78% 95% 33% Flatten:1, Success:323, ToDae:15 EMSL_TIMEOUT_MODEL_ATTEMPT:1
EQN_ALG_SIM 2 100% 50% 100% 0% Success:2 -
FUNC 1 100% 100% 100% 0% Success:1 -
OTHER 138 100% 67% 99% 16% Success:136, ToDae:2 -
Trace Accuracy vs OMC
MSL Package n Compared Trace High Near Channel OK No Severe
Blocks 32 47% 47% 31% 16% 97% 44%
Clocked 76 86% 80% 68% 12% 89% 84%
ComplexBlocks 2 50% 50% 50% 0% 100% 50%
Electrical.Analog 65 42% 37% 25% 12% 95% 34%
Electrical.Batteries 8 13% 13% 13% 0% 100% 13%
Electrical.Digital 23 96% 91% 91% 0% 85% 96%
Electrical.Machines 43 0% 0% 0% 0% 0% 0%
Electrical.Polyphase 5 40% 40% 0% 40% 98% 0%
Electrical.PowerConverters 59 49% 44% 15% 29% 96% 8%
Electrical.QuasiStatic.Machines 1 0% 0% 0% 0% 0% 0%
Electrical.QuasiStatic.Polyphase 4 0% 0% 0% 0% 0% 0%
Electrical.QuasiStatic.SinglePhase 6 0% 0% 0% 0% 0% 0%
Electrical.Spice3 14 7% 7% 7% 0% 100% 7%
Fluid 23 0% 0% 0% 0% 0% 0%
Magnetic.FluxTubes 20 5% 0% 0% 0% 67% 0%
Magnetic.FundamentalWave 27 0% 0% 0% 0% 0% 0%
Magnetic.QuasiStatic.FluxTubes 9 0% 0% 0% 0% 0% 0%
Magnetic.QuasiStatic.FundamentalWave 21 0% 0% 0% 0% 0% 0%
Math.FastFourierTransform 2 0% 0% 0% 0% 0% 0%
Math.Nonlinear 1 100% 100% 100% 0% 100% 100%
Math.Random 1 0% 0% 0% 0% 0% 0%
Mechanics.MultiBody 42 17% 14% 12% 2% 98% 14%
Mechanics.Rotational 17 53% 47% 41% 6% 98% 41%
Mechanics.Translational 16 63% 63% 63% 0% 100% 63%
Media 22 0% 0% 0% 0% 0% 0%
Media.Incompressible 1 0% 0% 0% 0% 0% 0%
StateGraph 7 57% 57% 43% 14% 100% 57%
Thermal.FluidHeatFlow 12 25% 25% 25% 0% 100% 25%
Thermal.HeatTransfer 4 75% 75% 50% 25% 99% 50%
Utilities 3 33% 33% 33% 0% 100% 33%
Overall 566 36% 33% 25% 8% 95% 29%

Speed vs OMC

Compilation and simulation are reported separately for the 185 trace-agreeing models with valid timings on both tools. Speedup = OMC / Rumoca (>1 means rumoca faster).

Aggregate speed summary
Phase Models Rumoca total (s) OMC total (s) Throughput speedup (×) Median per-model speedup (×)
Total 185 782.9 796.5 1.02 4.73
Compilation 185 298.5 737.9 2.47 5.50
Simulation 185 484.4 58.5 0.12 1.50
Speed breakdown
Where rumoca's time goes
Scalar eqns Models front→DAE Solve-IR + JIT integration
1–9 28 0.406 0.008 0.0010
10–24 32 0.494 0.010 0.0027
25–49 26 0.501 0.019 0.0086
50–99 44 0.548 0.163 0.0679
100–249 31 0.876 0.310 0.2815
250+ 24 1.658 2.440 12.3212
Rumoca stage Total seconds Share of rumoca total
front→DAE 143.1 18%
Solve-IR + JIT 155.4 20%
integration 484.4 62%
Speed by system size
Table 1 — Total (model → simulated results)

Rumoca = front-end compile + Solve-IR/JIT build + integration; OMC = timeTotal.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.4146 3.2441 6.60
10–24 32 0.5131 3.4367 6.93
25–49 26 0.6719 3.5022 5.95
50–99 44 0.8914 4.0671 4.63
100–249 31 1.7481 4.7536 2.79
250+ 24 16.6418 6.4552 0.37

Per-model speedup: min 0.12, median 4.73, max 9.98.

Table 2 — Compilation (build-to-runnable)

Rumoca = front-end compile + Solve-IR/JIT build; OMC = timeTotal - timeSimulation.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.4137 3.2291 6.60
10–24 32 0.5052 3.4172 6.92
25–49 26 0.5316 3.4738 6.46
50–99 44 0.7395 3.9283 5.34
100–249 31 1.2250 4.6272 3.77
250+ 24 4.0597 6.1202 1.20

Per-model speedup: min 0.19, median 5.50, max 9.99.

Table 3 — Simulation (integration only)

Rumoca = sim_run_seconds; OMC = timeSimulation.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.0010 0.0147 14.23
10–24 32 0.0027 0.0168 6.79
25–49 26 0.0086 0.0225 3.16
50–99 44 0.0679 0.0541 1.26
100–249 31 0.2815 0.0807 0.20
250+ 24 12.3212 0.2035 0.02

Per-model speedup: min 0.00, median 1.50, max 55.07.

Top 10 slowest models
Top 10 slowest rumoca compilation models

Trace-agreeing models ranked by rumoca time.

Model Scalar eqns Rumoca compile (s) OMC compile (s) Speedup (×)
Modelica.Mechanics.MultiBody.Examples.Elementary.FreeBody 1330 33.2676 6.3957 0.19
Modelica.Mechanics.MultiBody.Examples.Elementary.PointGravity 911 27.4765 9.4988 0.35
Modelica.Mechanics.MultiBody.Examples.Elementary.PendulumWithSpringDamper 1349 18.0898 8.7644 0.48
Modelica.Mechanics.MultiBody.Examples.Elementary.SpringMassSystem 1469 12.9942 5.6067 0.43
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_R 492 6.7777 6.5761 0.97
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.ThyristorBridge2mPulse_RL 497 6.3658 6.1240 0.96
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.ThyristorBridge2mPulse_R 491 6.0743 6.9514 1.14
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_RLV 504 5.9614 5.0518 0.85
Modelica.Mechanics.MultiBody.Examples.Elementary.Pendulum 934 5.7891 7.2329 1.25
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_RL 498 4.2798 6.3344 1.48
Top 10 slowest rumoca simulation models

Trace-agreeing models ranked by rumoca time.

Model Scalar eqns Rumoca simulation (s) OMC simulation (s) Speedup (×)
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2Pulse.ThyristorCenterTap2Pulse_RLV_Characteristic 180 34.5307 5.3315 0.15
Modelica.Mechanics.MultiBody.Examples.Elementary.PendulumWithSpringDamper 1349 30.2875 0.1898 0.01
Modelica.Electrical.PowerConverters.Examples.ACDC.Rectifier1Pulse.Thyristor1Pulse_R_Characteristic 121 29.5452 4.5589 0.15
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_R 492 27.5973 0.3572 0.01
Modelica.Mechanics.MultiBody.Examples.Elementary.PointGravity 911 24.6683 0.1254 0.01
Modelica.Mechanics.Rotational.Examples.OneWayClutch 58 24.6256 0.0943 0.00
Modelica.Electrical.PowerConverters.Examples.DCDC.ChopperStepDown.ChopperStepDown_R 134 24.3971 0.8446 0.03
Modelica.Mechanics.MultiBody.Examples.Elementary.FreeBody 1330 23.6589 0.1601 0.01
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_RLV 504 22.3312 0.2808 0.01
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.ThyristorBridge2mPulse_R 491 21.3566 0.3130 0.01

Additional Artifacts

  • msl_compatibility_report.md

@jgoppert
fix(msl): add tensor-aware compiler and solver support for MultiBody
5ad55c8 
Preserve record, function, array, and each-modifier semantics through instantiation, flattening, and DAE conversion.

Generalize derivative closure, state selection, index reduction, and structured scalar views while retaining compact tensor-native IR.

Add explicit initialization and algebraic projections, tensor JVPs, certified causal execution, FMI 3 event handling, and flake-pinned OMC parity gates.

The fixed 566-model MSL run reaches 188 acceptable OMC traces, up 38 from the promoted baseline, with MultiBody compilation retained.

Signed-off-by: James Goppert <james.goppert@gmail.com>
@jgoppert
jgoppert force-pushed the msl-trace-parity-recovery branch from a499eb8 to 5ad55c8 Compare July 21, 2026 12:39
@jgoppert jgoppert changed the title fix(msl): consolidate parity, MultiBody, FMI 3, and runtime performance fix(msl): add tensor-aware compiler and solver support for MultiBody Jul 21, 2026
@jgoppert
jgoppert merged commit ef5f071 into main Jul 21, 2026
41 checks passed
@jgoppert
jgoppert deleted the msl-trace-parity-recovery branch July 21, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jgoppert