Skip to content

Move heavy AD backends out of root [extras] into group envs#62

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:migrate-ad-group-env
Draft

Move heavy AD backends out of root [extras] into group envs#62
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:migrate-ad-group-env

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Please ignore until reviewed by @ChrisRackauckas. This is an automated draft.

What & why

FastPower's optional AD backends (Enzyme, ForwardDiff, Mooncake, ReverseDiff, Tracker) plus their test-only support deps (EnzymeTestUtils, StableRNGs) lived in the root Project.toml's [extras]/[targets].test. Under the folder-model harness both the Core group (top-level test/*.jl) and the Enzyme group folder run in the main test env, so this pulled the full joint AD graph into every root/Core resolve — including the Downgrade lane, which had to floor-resolve all five heavy backends together against the base package. That is exactly the joint-graph floor wall the group-env migration removes.

Each backend is moved into the group env whose files actually using it:

  • test/AD/Project.toml (new) — ForwardDiff/ReverseDiff/Tracker/Mooncake for test/AD/other_ad_engines_tests.jl. That file was a loose top-level Core file (using ForwardDiff, ReverseDiff, Tracker, Mooncake); it is relocated into its own AD group folder (declared in test_groups.toml) so those backends leave the root/Core env.
  • test/Enzyme/Project.toml (new) — Enzyme/EnzymeTestUtils/StableRNGs for the existing test/Enzyme/ group folder, which previously had no Project.toml and so resolved against the root env.

Root [extras]/[targets].test now keeps only SafeTestsets/SciMLTesting/Test (harness + Core deps). Weakdep [compat] entries (Enzyme, ForwardDiff, Mooncake, ReverseDiff, Tracker) stay because they still gate package extensions; the extra-only compats (EnzymeTestUtils, StableRNGs) are dropped so they don't dangle.

No runtests.jl change is needed — the folder-discovery harness (run_tests()) auto-activates each group folder's Project.toml (activate_group_env). The AD group is added to test/test_groups.toml, so the centralized grouped-tests.yml CI matrix picks it up automatically.

Downgrade effect (jdc-52 harness, --min=@deps, fresh depot each)

BEFORE (main): base resolves, then Independently downgrading 5 weakdep extension(s): Enzyme, ForwardDiff, Mooncake, ReverseDiff, Tracker — all five floor-checked in the root joint graph (writes Manifest-{Enzyme,ForwardDiff,Mooncake,ReverseDiff,Tracker}.toml). EnzymeTestUtils + StableRNGs also present in the base resolve.

AFTER (this branch): merged root project has only Test/SafeTestsets/SciMLTesting extras; base Successfully resolved minimal versions; no Independently downgrading … step at all — the root/Core Downgrade lane no longer floor-resolves any AD backend. No per-backend manifests written; base manifest contains none of the seven moved packages.

Verification (all run locally)

  • test/AD env: resolves + loads ForwardDiff 1.4.1 / ReverseDiff 1.17.0 / Tracker 0.2.38 / Mooncake 0.5.36; all FastPower*Ext precompile.
  • test/Enzyme env: resolves + loads Enzyme 0.13.173 / EnzymeTestUtils 0.2.8 / StableRNGs 1.0.4; FastPowerEnzymeExt precompiles.
  • GROUP=Core Pkg.test: passes (1200 + 5 assertions), root env carries no AD backend.
  • GROUP=AD Pkg.test: AD/other_ad_engines_tests.jl 4/4 via auto-activated env.
  • GROUP=Enzyme Pkg.test: enzyme_forward_tests.jl 52/52 + enzyme_reverse_tests.jl 36/36 via auto-activated env.

🤖 Generated with Claude Code

FastPower's optional AD backends (Enzyme, ForwardDiff, Mooncake,
ReverseDiff, Tracker) plus their test-only support deps (EnzymeTestUtils,
StableRNGs) were declared in the root Project.toml's [extras]/[targets].test.
Because the folder-model Core group and the Enzyme group folder both run in
the main test env, this pulled the full joint AD graph into every
root/Core resolve — including the Downgrade lane, which had to floor-resolve
all five backends together against the base package.

Move each backend into the group env whose files actually `using` it:

* test/AD/Project.toml (new) — ForwardDiff/ReverseDiff/Tracker/Mooncake for
  the relocated test/AD/other_ad_engines_tests.jl (was a loose top-level Core
  file, now its own AD group declared in test_groups.toml).
* test/Enzyme/Project.toml (new) — Enzyme/EnzymeTestUtils/StableRNGs for the
  existing test/Enzyme/ group folder (which previously had no Project.toml and
  so resolved against the root env).

Root [extras]/[targets].test keeps only SafeTestsets/SciMLTesting/Test (the
harness + Core deps). Weakdep [compat] entries (Enzyme, ForwardDiff, Mooncake,
ReverseDiff, Tracker) stay since they still gate package extensions; the
extra-only compats (EnzymeTestUtils, StableRNGs) are removed to avoid dangling.

Under the folder-discovery harness each group folder auto-activates its own
Project.toml, so no runtests.jl wiring changes are needed. The root/Core
Downgrade lane no longer floor-resolves any AD backend; cross-backend AD tests
are unchanged (AD 4/4, Enzyme 88/88, Core 1205/1205 all pass locally).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants