Skip to content

test: exercise SISO initialization through ODEProblem#491

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:agent/fix-siso-odeproblem-test
Draft

test: exercise SISO initialization through ODEProblem#491
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:agent/fix-siso-odeproblem-test

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jul 13, 2026

Copy link
Copy Markdown
Member

Please ignore this PR until reviewed by @ChrisRackauckas.

Summary

  • Construct the SISO test's initialized system through ODEProblem instead of solving the standalone InitializationProblem.
  • Keep both behavioral assertions exact and unchanged at 1.0.
  • Import ODEProblem from its declaring public module, SciMLBase, and remove the now-unused OrdinaryDiffEq import.

This is a focused extraction of the SISO change from draft PR #452. That PR contains unrelated changes and conflicts; none of those changes are included here. The branch is now rebased on current main after #489 merged, so this PR contains one commit and changes only test/utils.jl.

Root cause

PR #469 replaced the old manual generate_initializesystem / NonlinearProblem path with InitializationProblem. A standalone initialization problem is keyed to its intermediate initialization system, not the fully initialized original ODE system. This has two resolver-dependent failure modes:

  • Older minimum-compatible stack: initsol[sys.so.xd] is 0.0, not the declared 1.0 initial value.
  • Current downgrade stack: sys.so.u has been eliminated from the initialization system, so indexing it throws ArgumentError: Symbol so₊u(t) is not present in the system.

ODEProblem performs the complete operating-point / initialization preprocessing and retains the public indexing semantics for both original states and observed variables. The test still checks exactly the same two values; it now checks them on the object whose initialization behavior matters to users.

A signature-specific source bisect identifies 3672b875 (PR #469) as the first commit producing the missing-observed-symbol behavior; its parent a79a77f3 passes this SISO check on the preserved graph. git log --follow confirms #469 is still the latest main edit to this test.

Exact current-main reproduction

The current-main downgrade run 29870067371, job 88767752470, fails at test/utils.jl:41 with the missing-so₊u error: 13 pass / 1 error in Core/utils.jl.

I replayed exact main 4459350cf3dc5db67ab330ed9f2f8965a3aae1af and this one-commit candidate from fresh isolated depots with Julia 1.10.11. Both used exact action commit fab1defb76df9fd672f63c94df73ce131d32e134, the workflow's strict deps resolution, stdlib skip list, and Mooncake no-promote setting. All 171 installed package/version pairs match the hosted job, and the base/candidate manifests are byte-identical (SHA-256 fb0b7750f5a20e677cc23eb4e6ff3d8686829c95ee61bced6b6dcc399df0dc29).

  • Exact clean main: official GROUP=Core exited 1 with 587 pass / 3 broken / 1 error across 19 files, reproducing the hosted so₊u signature.
  • Exact candidate 8b156503: the same official GROUP=Core invocation exited 0 with 588 pass / 3 broken / 0 fail / 0 error, ending with Testing ModelingToolkitStandardLibrary tests passed.
  • Candidate Core/translational.jl: 16/16 passed, confirming the merged Initialize translational acceleration sensor test #489 fix is included through the base rather than duplicated here.
  • Candidate Core/utils.jl: 14/14 passed, including SISO 2/2 with both exact 1.0 assertions unchanged.
  • Runic 1.7.0 --check passed for the repository root.
  • git diff --check passed.

Earlier focused validation also passed the complete test/utils.jl on Julia 1.10.11 minimum/current graphs and Julia 1.12.6 current dependencies; the exact current-main full-harness replay above supersedes the former stacked-branch qualification.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Investigation scratchpad for commit 8bfa82b6:

  1. Reproduced the two resolver-specific clean-main failures without modifying source:
    • exact minimum stack: xd evaluated to 0.0 (SISO 1/2);
    • exact current-LTS stack: indexing the eliminated observed u threw ArgumentError (SISO 1 pass / 1 error).
  2. Checked prior art in test: SISO Check uses ODEProblem instead of generate_initializesystem #452, the superseded Use InitializationProblem() in the SISO tests #465, merged Use InitializationProblem() in the SISO tests (rebase of #465) #469, and ModelingToolkit issue Default values of variables are not converted to equations when building initialization systems. ModelingToolkit.jl#3917.
  3. Ran a signature-specific source bisect with the current-LTS dependency set. The predicate marked only the missing-observed-symbol error as bad and treated the predecessor's older non-square error as good for this signature. Result: 3672b875 (Use InitializationProblem() in the SISO tests (rebase of #465) #469) is the first bad commit.
  4. Replaced only the intermediate initialization solve with public SciMLBase.ODEProblem construction. The exact xd == 1.0 and u == 1.0 assertions were retained; no tolerance was changed and no test was skipped or disabled.
  5. Verified complete test/utils.jl exit 0 on the exact minimum stack, exact current-LTS stack, and Julia 1.12.6 current registry. Runic 1.7.0 checked all 81 tracked Julia files, and git diff --check passed.

The clean-main full Core run's earlier translational error is independently handled by #489; this commit deliberately does not duplicate that fix.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

CI classification: the Julia LTS Core failure is the independently audited clean-main translational regression, not this SISO change.

The SISO test was therefore not reached by this job. Its exact Julia 1.10.11 dependency set passed the complete test/utils.jl locally with this commit.

@ChrisRackauckas-Claude
ChrisRackauckas-Claude force-pushed the agent/fix-siso-odeproblem-test branch from 8bfa82b to 4c909e3 Compare July 13, 2026 22:35
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Stack update: #491 now contains exact #489 commit d698064b, followed by the rebased SISO commit 4c909e39.

Before the history rewrite I fetched both remote branches and verified:

I rebased only #491 and force-pushed it with an explicit lease against the verified old #491 hash. The #489 branch was not modified. Please merge #489 first; afterward this PR's effective diff collapses to test/utils.jl.

Post-stack local verification on Julia 1.10.11/current-LTS dependencies:

  • complete test/translational.jl: 16/16;
  • complete test/utils.jl: exit 0, SISO 2/2;
  • Runic 1.7.0 repository check: exit 0;
  • git diff --check: exit 0.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Independent audit adds two narrower boundary checks that complement the existing investigation:

  • On a preserved exact graph with ModelingToolkit 11.0.0, ModelingToolkitBase 1.1.0, Symbolics 7.4.1, SymbolicUtils 4.38.1, SciMLBase 2.125.0, and SciCompDSL 1.0.0, the post-Use InitializationProblem() in the SISO tests (rebase of #465) #469 standalone InitializationProblem path produced xd = 0.0, u = 1.0. Reconstructing the pre-Use InitializationProblem() in the SISO tests (rebase of #465) #469 explicit generate_initializesystem + mtkcompile + NonlinearProblem path on that same graph produced xd = 1.0, u = 1.0. Thus the earlier manual path is healthy on this minimum graph even though the bisect notes its separate non-square failure on the current-LTS graph.
  • The public empty-vector form ODEProblem(sys, [], (0.0, 1.0)) produced xd = 1.0, u = 1.0 on that exact minimum graph, and the complete patched test/utils.jl passed 8/8 on both Julia 1.10.11 and 1.12.6 current graphs. This independently confirms that the original exact assertions work through the public ODE problem path without depending on standalone initialization-system indexing.

No additional code change is suggested by this comment.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Replacement stacked CI confirms that the Julia LTS Core suite passes end to end:

  • Job: Tests - Core (Julia lts)
  • Core/translational.jl: 16/16 passed.
  • Core/utils.jl: 14/14 passed.
  • ModelingToolkitStandardLibrary package tests passed.

This verifies the #489 translational fix and this PR's SISO fix together on Julia LTS. PR #491 remains stacked on the exact #489 head commit so the two circular CI failures can be reviewed and merged in order.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

The stacked Downgrade Tests job also passes end to end:

  • Core/translational.jl: 16/16 passed.
  • Core/utils.jl: 14/14 passed.
  • ModelingToolkitStandardLibrary package tests passed.

This is the dependency lane that previously exposed the lost xd default (0.0 instead of 1.0) in the standalone InitializationProblem; the ODEProblem assertion now passes there without changing its expected value or tolerance.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Combined-stack CI update: Core passes with both commits present on Julia LTS, current, and prerelease. QA, downgrade, documentation, Runic, and spelling also pass. The only remaining active check is ModelingToolkit downstream. An independent fresh-depot Julia 1.12 GROUP=Core run of this exact two-commit head is still running locally; I will post its native summary separately rather than treating CI as local verification.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Independent local combined-stack validation on exact head 4c909e3: the native GROUP=Core Pkg.test run under Julia 1.12 completed with exit 0 and the final ModelingToolkitStandardLibrary tests-passed message. The target files passed Core/translational.jl 16/16 and Core/utils.jl 14/14 in the same full Core process. Whole-checkout Runic then exited 0 and git diff --check exited 0. Transparency note: an initial Runic-dot invocation traversed the untracked .depot package cache and hit Runic parser assertions in MLStyle/SymbolicUtils dependency test files; moving that temporary cache outside the checkout produced the repository-only exit-0 result above.

The standalone initialization problem represents an intermediate system, so it can lose defaults or observed variables from the original ODE system. Construct the ODE problem and assert the same initialized values through its public indexing API.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude
ChrisRackauckas-Claude force-pushed the agent/fix-siso-odeproblem-test branch from 4c909e3 to 8b15650 Compare July 21, 2026 22:58
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Current-main rebase and exact downgrade replay:

  • Initialize translational acceleration sensor test #489 is now merged, so I fetched current main (4459350c) and the exact remote PR branch before rewriting. The remote contained only the two previously documented commits by ChrisRackauckas-Claude; no outside commits were present.
  • I rebased only the SISO change as commit 8b156503 and force-pushed with an explicit lease against verified old head 4c909e39. The resulting PR is one commit / one file (test/utils.jl) ahead of current main, with the co-author trailer intact.
  • The hosted current-main failure is run 29870067371, job 88767752470: ArgumentError: Symbol so₊u(t) is not present, Core/utils.jl 13 pass / 1 error.
  • Fresh local base and candidate runs used Julia 1.10.11, exact downgrade action commit fab1defb, strict deps, and byte-identical manifests. Their 171 installed package/version pairs also match the hosted job exactly.
  • Exact base: 587 pass / 3 broken / 1 error, reproducing the hosted SISO signature.
  • Exact candidate: 588 pass / 3 broken / 0 fail / 0 error across all 19 Core files; Core/utils.jl 14/14 and package tests passed.
  • Whole-repository Runic 1.7.0 check and git diff --check both exited 0.

The PR body now reflects the unstacked current-main state and these fresh controls.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Post-hold terminal CI classification for current head 8b156503:

  • All 11 hosted checks are terminal: 10 passed, 1 cancelled, 0 pending.
  • The target Downgrade Tests check passed, as did Core on Julia lts/current/pre, QA, docs, Runic, suggestions, spelling, and detection.
  • The only non-success is ModelingToolkit.jl/All / Downstream Tests - All. GitHub cancelled it after the explicit six-hour job limit. Its sole annotation is The job has exceeded the maximum execution time of 6h0m0s.
  • The downstream test step was still marked in_progress; coverage and cleanup steps never started. GitHub's job-log endpoint returns BlobNotFound, so there is no Julia exception or failing assertion to attribute to this PR.

The exact local minimum graph remains decisive for this one-test change: clean base reports 587 pass / 3 existing broken / 1 SISO error, while 8b156503 reports 588 pass / 3 existing broken / 0 failures or errors with the same 171 package/version pairs and byte-identical manifest. Repository Runic and the focused current/minimum SISO tests also pass.

I am therefore classifying the lone cancellation as the downstream All job's six-hour execution ceiling, not a head-specific test failure. No test was skipped, weakened, or silenced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants