Skip to content

refactor(ir): compact structured array families#326

Merged
jgoppert merged 1 commit into
mainfrom
array-native-compiler
Jul 21, 2026
Merged

refactor(ir): compact structured array families#326
jgoppert merged 1 commit into
mainfrom
array-native-compiler

Conversation

@jgoppert

@jgoppert jgoppert commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Branch Naming

  • array-native-compiler is descriptive and has no agent/ prefix.

Summary

  • Replace cardinality-sized equation_counts vectors in Flat and DAE structured families with one uniform equations_per_point value. The DAE JSON schema moves from version 7 to 8.
  • Add checked, lazy row-major domain algebra (extents, ordinal strides, ordinal/tuple conversion, corner ordinals, and exact lazy iteration) so compact paths no longer need to enumerate a Cartesian domain.
  • Use compact domain arithmetic in regular-family structural incidence and Solve tensor lowering. Large regular families now inspect only O(rank) representative rows; the bounded scalar-view cross-check remains a correctness oracle for small families.
  • Add stable tensor-preservation counters and fallback reason codes so tests and profiling can budget scalarization without materializing the domain. The million-point regression derives its two-million-row fallback count in constant domain space.
  • This is the review's P0/P1 ownership and observability milestone. It does not claim the later compressed incidence/matching/BLT redesign: structural matching still owns scalar incidence sets, but no longer materializes index tuples merely to recover compact-domain shape.

Spec / MLS Alignment

  • Relevant active specs checked: SPEC_0007, SPEC_0008, SPEC_0021, SPEC_0025, SPEC_0029, SPEC_0032 development process, and SPEC_0032 range-preserving tensors.
  • Relevant MLS sections: none; Modelica language semantics are unchanged. This changes compiler representation, domain algebra, and lowering observability.
  • Crate/phase owners: rumoca-core owns shared structured-domain algebra; Flat/DAE own family representation; flatten owns family construction; structural owns incidence derivation; Solve owns tensor preservation/fallback reporting; codegen consumes the versioned DAE shape.

Risk and Design Notes

  • Main correctness risk: formula-based ordinal math or corner selection could disagree with the existing deterministic scalar-view order. Reverse/non-unit/empty ranges, millionth-ordinal lookup, focused incidence/stencil tests, bounded full-row equivalence checks, and the workspace suite cover that contract.
  • Main maintenance risk: the Flat/DAE schema field is intentionally breaking. The schema version is bumped to 8 and all in-tree consumers moved together; no old/new compatibility path remains.
  • Why the change belongs in these crates: domain formulas are shared foundation behavior, while construction, remapping, incidence, tensor lowering, and rendering remain in their existing phase owners.
  • New public API: checked compact-domain operations, DAE row-count helpers, and a Solve tensor-preservation report. cargo-public-api reports 32 added and 3 removed items, including public struct fields and the deliberate replacement of equation_counts.
  • Remaining review milestone: compressed structural incidence, matching, and BLT should follow as a separate measured redesign using these counters. This PR does not hide that scalar graph cost behind a tensor-native claim.

Testing

  • nix develop -c cargo fmt --check: pass.
  • nix develop -c cargo clippy --workspace --all-targets --all-features -- -D warnings: pass.
  • nix develop -c nix shell nixpkgs#libxml2 -c cargo test --workspace: pass. The ordinary dev shell initially exposed two GALEC test failures because xmllint was absent; supplying libxml2 made the complete workspace suite pass.
  • nix develop -c cargo doc --workspace --no-deps: pass.
  • Focused core, DAE, flatten/DAE/structural/Solve/codegen, CLI JSON, and GALEC projection suites: pass.
  • The required MSL command passed under the checked-in parity configuration: all 2,555 files parsed; the configured 10-model MultiBody cohort compiled and balanced 10/10 and simulated 8/10. The two known structural-singularity simulations remain DoublePendulum and SpringWithMass.
  • The local MSL run was a committed focused target scope, so the harness correctly skipped fixed-denominator baseline-delta promotion. The draft still requires the full sharded CI baseline gate before merge.
  • ModelicaTest and pinned modelica_models semantic gates were not run locally because this PR does not change MLS semantics.

Code Size Budget (required)

  • production_lines_added: 564
  • production_lines_deleted: 280
  • test_lines_added: 200
  • test_lines_deleted: 68
  • public_items_added: 32
  • public_items_removed: 3
  • files_touched: 25
  • net_added_lines: 416

The positive growth is the reusable checked domain algebra, stable preservation report, and regressions that prove large-domain behavior without enumeration. The first compression pass removed the recursive Cartesian tuple builder, per-point family vectors and remapping vectors, codegen's sequence-field parser, and structural tuple/set-based extent recovery. The next compression milestone is the review's separate compressed incidence/matching/BLT redesign; it should consume the new measurements rather than introduce a parallel scalar/tensor ownership path. No separate tracking issue is created by this draft.

Reviewer Checklist

  • Relevant active specs were checked.
  • MLS-sensitive changes cite the right MLS section (not applicable; semantics unchanged).
  • Crate boundaries and phase ownership preserved (SPEC_0029).
  • Tests prove behavior and disclose the remaining compressed-structural gap.
  • Standard local gates pass (fmt, clippy -D warnings, workspace tests, docs).
  • Full fixed-denominator MSL baseline gate passes in CI; the configured focused local gate passed as described above.
  • Size-budget section completed.
  • Positive net diff has explicit compression justification.
  • New APIs are required for compact-domain reuse and measurable preservation.
  • Old cardinality-sized metadata and tuple-recovery paths were removed rather than retained in parallel.
  • No #[allow(clippy::...)] added outside generated code.
  • The commit is signed off (git commit -s) with no Co-Authored-By trailer.
  • No external implementation material was added.

@jgoppert
refactor(ir): compact structured array families
6eb8729 
Signed-off-by: James Goppert <james.goppert@gmail.com>
@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown

MSL Quality Gate Summary

Generated by cargo xtask repo msl pr-comment from CI result artifacts.

Scope MSL OMC Compile Balance Initial Balance Simulation
full v4.1.0 a96aa1a-cmake 545/532 (Δ0/0) 532/532 (Δ0/0) 532/532 (Δ0/0) 206/566 (Δ+36/0)

Deltas compare numerator/denominator against the resolved MSL quality baseline.
Baseline: quality gate v1, commit 08fac54, MSL v4.1.0, OMC OpenModelica 1.27.0~dev.beta.3.
Trace agreement vs baseline: high+near Δ+37, deviation Δ-2.

CI Gate Snapshot

These are the baseline-relative MSL stats checked by CI; IC progress is contextual unless simulation successes regress, and speed metrics are informational only.

Gate area Current (Δ vs baseline)
Structural floors parse 566 (0), flat 565 (0), DAE 545 (0), solve 444 (+63)
Balance floors balance 532/532 (0), init 532/532 (0)
Runtime gate IC progress 252/566 (+13), sim 206/566 (+36)
Trace floors acceptable 187/566 (+37), no-severe 164/566 (+15), compared 201 (+35)
CI gate details
CI gate metric Current Baseline Δ
Parse floor 566/566 566/566 0
Flatten floor 565/566 565/566 0
DAE/compile floor 545/566 545/566 0
Solve-IR floor 444/566 381/566 +63
Balance floor 532/532 532/532 0
Initial balance floor 532/532 532/532 0
IC progress 252/566 239/566 +13
Simulation floor 206/566 170/566 +36
Partial models ceiling 13 13 0
Unbalanced models ceiling 0 0 0
Trace acceptable floor 187/566 150/566 +37
Trace no-severe floor 164/566 149/566 +15
Trace compared floor (drop ≤2) 201 166 +35

Package Pass Rates

MSL Package n Ast Flat Dae Solve IC Sim
Overall 566 100% 100% 96% 81% 39% 33%
Per-package pass rates
MSL Package n Ast Flat Dae Solve IC Sim
Blocks 32 100% 100% 97% 84% 41% 47%
Clocked 76 100% 100% 100% 100% 82% 80%
ComplexBlocks 2 100% 100% 100% 100% 50% 50%
Electrical.Analog 65 100% 100% 100% 88% 57% 37%
Electrical.Batteries 8 100% 100% 50% 50% 13% 13%
Electrical.Digital 23 100% 100% 100% 100% 96% 91%
Electrical.Machines 43 100% 100% 100% 98% 0% 0%
Electrical.Polyphase 5 100% 100% 100% 100% 60% 40%
Electrical.PowerConverters 59 100% 100% 100% 76% 39% 42%
Electrical.QuasiStatic.Machines 1 100% 100% 100% 100% 0% 0%
Electrical.QuasiStatic.Polyphase 4 100% 100% 100% 100% 0% 0%
Electrical.QuasiStatic.SinglePhase 6 100% 100% 100% 100% 0% 0%
Electrical.Spice3 14 100% 100% 93% 79% 7% 7%
Fluid 23 100% 100% 57% 4% 0% 0%
Magnetic.FluxTubes 20 100% 100% 100% 95% 15% 0%
Magnetic.FundamentalWave 27 100% 100% 100% 4% 0% 0%
Magnetic.QuasiStatic.FluxTubes 9 100% 100% 100% 100% 0% 0%
Magnetic.QuasiStatic.FundamentalWave 21 100% 100% 95% 86% 0% 0%
Math.FastFourierTransform 2 100% 100% 100% 100% 0% 0%
Math.Nonlinear 1 100% 100% 100% 100% 100% 100%
Math.Random 1 100% 100% 100% 100% 0% 0%
Mechanics.MultiBody 42 100% 100% 100% 83% 14% 14%
Mechanics.Rotational 17 100% 100% 100% 100% 88% 47%
Mechanics.Translational 16 100% 100% 100% 100% 81% 63%
Media 22 100% 100% 100% 59% 5% 0%
Media.Incompressible 1 100% 100% 100% 0% 0% 0%
StateGraph 7 100% 100% 57% 57% 57% 57%
Thermal.FluidHeatFlow 12 100% 100% 100% 100% 75% 25%
Thermal.HeatTransfer 4 100% 100% 100% 100% 75% 75%
Utilities 3 100% 67% 67% 67% 67% 33%
MLS Contract Coverage
MLS Category n Compile Solve IR Balance Sim Phases Errors
ARR 9 89% 89% 89% 89% Flatten:1, Success:8 -
CLK_SM 77 100% 96% 96% 84% Success:74, ToDae:3 -
CONN_STRM 339 100% 79% 96% 33% Success:324, ToDae:15 -
EQN_ALG_SIM 2 100% 50% 100% 0% Success:2 -
FUNC 1 100% 100% 100% 0% Success:1 -
OTHER 138 100% 67% 99% 16% Success:136, ToDae:2 -
Trace Accuracy vs OMC
MSL Package n Compared Trace High Near Channel OK No Severe
Blocks 32 47% 47% 31% 16% 97% 44%
Clocked 76 86% 80% 68% 12% 89% 84%
ComplexBlocks 2 50% 50% 50% 0% 100% 50%
Electrical.Analog 65 42% 37% 25% 12% 95% 34%
Electrical.Batteries 8 13% 13% 13% 0% 100% 13%
Electrical.Digital 23 96% 91% 91% 0% 85% 96%
Electrical.Machines 43 0% 0% 0% 0% 0% 0%
Electrical.Polyphase 5 40% 40% 0% 40% 98% 0%
Electrical.PowerConverters 59 47% 42% 14% 29% 95% 8%
Electrical.QuasiStatic.Machines 1 0% 0% 0% 0% 0% 0%
Electrical.QuasiStatic.Polyphase 4 0% 0% 0% 0% 0% 0%
Electrical.QuasiStatic.SinglePhase 6 0% 0% 0% 0% 0% 0%
Electrical.Spice3 14 7% 7% 7% 0% 100% 7%
Fluid 23 0% 0% 0% 0% 0% 0%
Magnetic.FluxTubes 20 5% 0% 0% 0% 67% 0%
Magnetic.FundamentalWave 27 0% 0% 0% 0% 0% 0%
Magnetic.QuasiStatic.FluxTubes 9 0% 0% 0% 0% 0% 0%
Magnetic.QuasiStatic.FundamentalWave 21 0% 0% 0% 0% 0% 0%
Math.FastFourierTransform 2 0% 0% 0% 0% 0% 0%
Math.Nonlinear 1 100% 100% 100% 0% 100% 100%
Math.Random 1 0% 0% 0% 0% 0% 0%
Mechanics.MultiBody 42 17% 14% 12% 2% 98% 14%
Mechanics.Rotational 17 53% 47% 41% 6% 98% 41%
Mechanics.Translational 16 63% 63% 63% 0% 100% 63%
Media 22 0% 0% 0% 0% 0% 0%
Media.Incompressible 1 0% 0% 0% 0% 0% 0%
StateGraph 7 57% 57% 43% 14% 100% 57%
Thermal.FluidHeatFlow 12 25% 25% 25% 0% 100% 25%
Thermal.HeatTransfer 4 75% 75% 50% 25% 99% 50%
Utilities 3 33% 33% 33% 0% 100% 33%
Overall 566 36% 33% 25% 8% 95% 29%

Speed vs OMC

Compilation and simulation are reported separately for the 184 trace-agreeing models with valid timings on both tools. Speedup = OMC / Rumoca (>1 means rumoca faster).

Aggregate speed summary
Phase Models Rumoca total (s) OMC total (s) Throughput speedup (×) Median per-model speedup (×)
Total 184 751.5 766.2 1.02 4.77
Compilation 184 312.8 717.3 2.29 5.47
Simulation 184 438.7 49.0 0.11 1.83
Speed breakdown
Where rumoca's time goes
Scalar eqns Models front→DAE Solve-IR + JIT integration
1–9 28 0.379 0.008 0.0009
10–24 32 0.507 0.011 0.0027
25–49 26 0.531 0.018 0.0070
50–99 44 0.576 0.142 0.0660
100–249 30 0.896 0.337 0.4472
250+ 24 1.614 2.551 9.6204
Rumoca stage Total seconds Share of rumoca total
front→DAE 142.9 19%
Solve-IR + JIT 169.9 23%
integration 438.7 58%
Speed by system size
Table 1 — Total (model → simulated results)

Rumoca = front-end compile + Solve-IR/JIT build + integration; OMC = timeTotal.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.3864 3.0610 7.71
10–24 32 0.5233 3.2865 6.54
25–49 26 0.6074 3.4471 5.68
50–99 44 0.8446 3.7400 4.38
100–249 30 1.9663 4.5492 2.32
250+ 24 12.9240 6.1372 0.41

Per-model speedup: min 0.11, median 4.77, max 9.84.

Table 2 — Compilation (build-to-runnable)

Rumoca = front-end compile + Solve-IR/JIT build; OMC = timeTotal - timeSimulation.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.3856 3.0458 7.70
10–24 32 0.5190 3.2702 6.56
25–49 26 0.5547 3.3718 5.96
50–99 44 0.7296 3.6133 5.01
100–249 30 1.2979 4.3159 3.30
250+ 24 3.7470 5.8350 1.44

Per-model speedup: min 0.19, median 5.47, max 9.82.

Table 3 — Simulation (integration only)

Rumoca = sim_run_seconds; OMC = timeSimulation.

Scalar eqns Models Rumoca (s) OMC (s) Speedup (×)
1–9 28 0.0009 0.0149 15.10
10–24 32 0.0027 0.0168 7.10
25–49 26 0.0070 0.0215 2.92
50–99 44 0.0660 0.0488 1.22
100–249 30 0.4472 0.0749 0.24
250+ 24 9.6204 0.2058 0.02

Per-model speedup: min 0.00, median 1.83, max 90.81.

Top 10 slowest models
Top 10 slowest rumoca compilation models

Trace-agreeing models ranked by rumoca time.

Model Scalar eqns Rumoca compile (s) OMC compile (s) Speedup (×)
Modelica.Mechanics.MultiBody.Examples.Elementary.FreeBody 1330 42.6975 8.2112 0.19
Modelica.Mechanics.MultiBody.Examples.Elementary.PointGravity 911 29.9306 8.4728 0.28
Modelica.Mechanics.MultiBody.Examples.Elementary.PendulumWithSpringDamper 1349 18.1695 7.8398 0.43
Modelica.Mechanics.MultiBody.Examples.Elementary.SpringMassSystem 1469 17.1818 7.3593 0.43
Modelica.Mechanics.MultiBody.Examples.Elementary.PointGravityWithPointMasses 1113 11.8018 6.3527 0.54
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_RLV 504 7.0534 6.4614 0.92
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_R 492 6.4390 5.8299 0.91
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.HalfControlledBridge2mPulse 464 5.7925 6.0431 1.04
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.ThyristorBridge2mPulse_R 491 5.6872 6.3088 1.11
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.ThyristorBridge2mPulse_RL 497 5.6576 5.8402 1.03
Top 10 slowest rumoca simulation models

Trace-agreeing models ranked by rumoca time.

Model Scalar eqns Rumoca simulation (s) OMC simulation (s) Speedup (×)
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_RLV 504 35.9667 0.3884 0.01
Modelica.Mechanics.MultiBody.Examples.Elementary.FreeBody 1330 34.7939 0.2298 0.01
Modelica.Mechanics.MultiBody.Examples.Elementary.PendulumWithSpringDamper 1349 31.0550 0.1653 0.01
Modelica.Mechanics.MultiBody.Examples.Elementary.PointGravity 911 27.3910 0.1083 0.00
Modelica.Electrical.PowerConverters.Examples.DCDC.ChopperStepDown.ChopperStepDown_RL 140 25.3269 0.7567 0.03
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierCenterTap2mPulse.ThyristorCenterTap2mPulse_R 492 24.0146 0.3106 0.01
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.HalfControlledBridge2mPulse 464 23.8932 0.4198 0.02
Modelica.Mechanics.Rotational.Examples.OneWayClutch 58 21.7963 0.0794 0.00
Modelica.Electrical.PowerConverters.Examples.ACDC.RectifierBridge2mPulse.ThyristorBridge2mPulse_R 491 20.9626 0.2633 0.01
Modelica.Electrical.PowerConverters.Examples.DCDC.ChopperStepDown.ChopperStepDown_R 134 16.5646 0.6497 0.04

Additional Artifacts

  • msl_compatibility_report.md

@jgoppert
jgoppert marked this pull request as ready for review July 21, 2026 17:59
@jgoppert
jgoppert merged commit 928348f into main Jul 21, 2026
41 checks passed
@jgoppert
jgoppert deleted the array-native-compiler branch July 21, 2026 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jgoppert