Shard sublibrary-project-tests matrix under GitHub's 256-job cap#107
Merged
ChrisRackauckas merged 1 commit intoJul 5, 2026
Conversation
The affected-sublibrary test matrix is fed into a single strategy.matrix on a job that calls a reusable workflow (uses: tests.yml). GitHub caps a matrix at 256 jobs; above that it instantiates no cells and fails the run with nothing scheduled. A monorepo-wide change that touches many sublibraries at once (e.g. a QA/ExplicitImports conversion editing every lib/*/test/qa) can push the affected set past 256 (OrdinaryDiffEq.jl #3777 computed 266 cells, incl. 56 QA cells), so the whole test job silently produced zero jobs and QA never ran. Split the computed projects-matrix into 200-cell shards in detect and run one test-N reusable-workflow job per shard, each with has_N gating. Small affected sets are unchanged (everything lands in shard_1; shards 2-4 skip). A >800-cell total errors loudly asking for another shard rather than silently dropping jobs. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
sublibrary-project-tests.ymlfeeds the entire affected-sublibrary matrix into a singlestrategy.matrixon a job that calls a reusable workflow (uses: tests.yml@v1). GitHub caps a matrix at 256 jobs; above that it instantiates no cells and fails the run with nothing scheduled (no annotation, onlydetectshows up as a job).A monorepo-wide change that touches many sublibraries at once trips this. Concretely, OrdinaryDiffEq.jl #3777 (a QA/ExplicitImports conversion editing ~54
lib/*/test/qa/*) madedetectcompute a 266-cell projects-matrix (incl. 56 QA cells).detectsucceeded andhas_changes=true, but thetestjob produced zero jobs and the run failed — so QA never ran for any sublibrary, silently. Smaller PRs (e.g. an 87-cell master push) spawn fine, including their QA cells, which is why this went unnoticed.Fix
Split the computed projects-matrix into 200-cell shards in
detectand run onetest-Nreusable-workflow job per shard (4 shards → 800-cell capacity), each gated byhas_N. Every per-jobstrategy.matrixnow stays well under 256.shard_1; shards 2–4 skip viahas_N == 'false'.>800-cell total errors loudly asking for another shard, rather than silently dropping jobs.Validation (local)
Replayed the exact detect shell logic against the real 266-cell #3777 matrix (extracted from the failing run's detect log):
Also verified the 4-cell (single-sublib) and empty-matrix cases behave as before. YAML parses. No external consumer references this workflow's removed
matrix/has_changesoutputs (grouped-tests.ymlhas its own separatedetect).Takes effect for OrdinaryDiffEq #3777 (and every monorepo using
@v1) once merged andv1is retagged.Please ignore until reviewed by @ChrisRackauckas.
🤖 Generated with Claude Code