ci: intersect flat --solvers list with each problem in the benchmark matrix#94
Open
andrinr wants to merge 1 commit into
Open
ci: intersect flat --solvers list with each problem in the benchmark matrix#94andrinr wants to merge 1 commit into
andrinr wants to merge 1 commit into
Conversation
benchmark-run.yml runs a matrix over problems and passed the same flat inputs.solvers list to every leg's `mosaic run`. A cross-domain list (e.g. deal.II,FEniCS,Firedrake,JAX-FEM,torch-fem) then fails on the structural-mesh leg, because torch-fem is thermal-only and `mosaic run` rejects a solver name absent from the problem it runs against (TopOpt.jl is structural's torch-fem counterpart). One bad leg fails the whole dispatch. Add .github/scripts/intersect-solvers.py and, in the Run step, intersect the requested solvers with the current problem's own solver set before invoking mosaic; skip the leg when the intersection is empty. Each leg now runs exactly the requested solvers it actually has. The pull-solver-images.py step already skips out-of-problem solvers, so only the run step needed this. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
📊 View the full benchmark resultsNo benchmarks ran for this PR, so there is no status report. |
Contributor
|
@andrinr Not quite understanding what bug is being fixed here. How can I trigger the error above? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
benchmark-run.ymlruns a matrix over problems and applied the same flatinputs.solverslist to every leg'smosaic run(--problems "${{ matrix.problem }}" --solvers "$SOLVERS"). A cross-domain list breaks the legs whose problem lacks some of those solvers:torch-femis thermal-only (structural-mesh hasTopOpt.jlinstead), andmosaic run(_helpers.py:471) rejects a name absent from the problem it runs against. One bad leg fails the whole dispatch — which is exactly what blocks verifying a thermal-only change (e.g. #85) whenever the matrix also includes structural-mesh.Fix
.github/scripts/intersect-solvers.py— prints the intersection of a flat solver list with one problem's own solver set (case-insensitive, canonical casing preserved), mirroring the existingfilter-problems-by-solver.py.SOLVERSwithmatrix.problembefore callingmosaic run; skip the leg when the intersection is empty.Each leg now runs exactly the requested solvers it actually has:
deal.II,FEniCS,Firedrake,JAX-FEM,torch-femFEniCS,Firedrake,JAX-FEM,deal.II(torch-fem dropped)::notice::The
pull-solver-images.pystep already skips out-of-problem solvers (pull-solver-images.py:96), so only the run step needed this.Verification
intersect-solvers.pycompiles; YAML validates; the intersection logic is unit-tested (cases above).🤖 Generated with Claude Code