fix(build-tools): prevent default ^* deps on subtasks of configured groups#26431
fix(build-tools): prevent default ^* deps on subtasks of configured groups#26431tylerbutler wants to merge 5 commits intomicrosoft:mainfrom
Conversation
…roups Named subtasks created by concurrently (e.g. check:exports:cjs:alpha) were receiving the default after: ["^*"] dependency independently, making them depend on ALL upstream tasks. This caused unrelated changes (like touching an eslint config) to invalidate downstream api-extractor tasks. Skip default 'after' processing for subtasks whose parent group has explicit configuration, since those subtasks already inherit the correct dependencies via initializeDependentLeafTasks.
Workaround for the default after: ["^*"] issue. Adds explicit task definitions for all check:exports:* subtask names so they depend on "api" instead of inheriting the default "^*" dependency on all upstream tasks.
Replace single parentTask pointer with precomputed Sets of subtasks partitioned by parent config type. A task's default ^* is only skipped if ALL its parent groups have explicit config (i.e. it does not appear in any default-config group). This handles the edge case where a cached task is a subtask of multiple GroupTasks with different config types.
| // Collect named subtasks of group tasks, partitioned by whether the parent | ||
| // group has explicit (non-default) configuration. A task can be a subtask of | ||
| // multiple groups (since getScriptTask returns cached instances), so we track | ||
| // both sets to handle that correctly: default 'after' is only skipped for a | ||
| // subtask if it appears in an explicit group and NOT in any default group. |
There was a problem hiding this comment.
This implies we are iterating over group tasks, but it looks like we are iterating over all tasks. More comments would be helpful.
| // Workaround: subtasks created by concurrently in "check:exports" receive the default | ||
| // after: ["^*"] dependency, making them depend on ALL upstream tasks (e.g. check:biome). | ||
| // Explicit definitions here override the default so they only depend on "api". | ||
| // "check:exports:bundle-release-tags" is handled per-package with ["build:esnext"]. | ||
| "check:exports:cjs:alpha": ["api"], | ||
| "check:exports:cjs:beta": ["api"], | ||
| "check:exports:cjs:EditLog": ["api"], | ||
| "check:exports:cjs:index": ["api"], | ||
| "check:exports:cjs:indexBrowser": ["api"], | ||
| "check:exports:cjs:indexNode": ["api"], | ||
| "check:exports:cjs:legacy": ["api"], | ||
| "check:exports:cjs:legacy.alpha": ["api"], | ||
| "check:exports:cjs:legacyAlpha": ["api"], | ||
| "check:exports:cjs:public": ["api"], | ||
| "check:exports:cjs:testUtils": ["api"], | ||
| "check:exports:esm:alpha": ["api"], | ||
| "check:exports:esm:beta": ["api"], | ||
| "check:exports:esm:browser:current": ["api"], | ||
| "check:exports:esm:browser:legacy": ["api"], | ||
| "check:exports:esm:EditLog": ["api"], | ||
| "check:exports:esm:index": ["api"], | ||
| "check:exports:esm:indexBrowser": ["api"], | ||
| "check:exports:esm:indexNode": ["api"], | ||
| "check:exports:esm:legacy": ["api"], | ||
| "check:exports:esm:legacy.alpha": ["api"], | ||
| "check:exports:esm:legacyAlpha": ["api"], | ||
| "check:exports:esm:node:current": ["api"], | ||
| "check:exports:esm:node:legacy": ["api"], | ||
| "check:exports:esm:public": ["api"], | ||
| "check:exports:esm:testUtils": ["api"], | ||
| "check:exports:index": ["api"], |
There was a problem hiding this comment.
If this workaround is kept, the specific api-extractor:commonjs|esnext tasks should be used to avoid cross cjs-esm dependencies.
| "check:exports:esm:node:legacy": ["api"], | ||
| "check:exports:esm:public": ["api"], | ||
| "check:exports:esm:testUtils": ["api"], | ||
| "check:exports:index": ["api"], |
There was a problem hiding this comment.
This check:exports:index setting is not correct. That package does not have an "api" task. If I understand correctly, the transitive dependency on "api-extractor:esnext" becomes nothing because there is no "api-extractor:esnext" script entry.
A correction proposed in #26442. (The PR also addresses "check:exports:bundle-release-tags" that is only sometimes configured in packages.)
| const after = expandStar(taskConfig.after, getAfterStarTaskNames); | ||
| traceTaskDepTask( | ||
| `Expanding ${taskConfig.isDefault ? "default " : ""}after: ${ | ||
| task.nameColored | ||
| } -> ${JSON.stringify(after)}`, | ||
| ); |
There was a problem hiding this comment.
It feels like applying such a default should issue a warning if not an error.
I would love to get to a place where every interdependent task has those configured somewhere. Should be possible for policy checker to find these and make changes. (A fully automated understanding is great. We should get that understanding captured into dependency listings that are then human visible without asking the tooling what the deps are.)
There was a problem hiding this comment.
I agree in principle. I just don't think our build should be so complicated that we need such level of automated checks. When I investigated the latest problems, I was surprised at just how many invocations of api extractor we have and how complex some of the individual package builds are now. Seems like there's room for better guidance on adding build tasks. I also underestimated how much adding new tasks would be a thing folks needed to do often.
I wonder if a simple rule might help, like "if your package can't use the default tasks, then you have to define all its tasks directly in the package itself instead of inheriting some and not inheriting others." That is, if you have a fluid build entry in the package.json, it describes that package's build graph entirely. Then it is easier to reason about what a package's build graph looks like. It either looks like the default graph based on the tasks defined, or it has a custom config that is self contained but duplicative.
Ultimately, I think we should be constructing our tasks such that we can swap fluid-build for something else with little fuss. That's one reason I've been pushing for less code in fluid build and more config level task definitions and behavior. It's easier to map such things to systems like Nx.
If we reach a certain point of build complexity, I don't think a task oriented system will scale enough. I think we'll need to look at more formal build-output based systems like bazel.
But the biggest reason I want to get off of fluid build is that if we were on another system, there would ideally be docs that described the expected behavior and how to config to the way we want it, while now that answer is "ask Tyler or read the code if he doesn't know" (likely). :)
There was a problem hiding this comment.
I can put the behavior change to error here behind an env var and we can try it out.
There was a problem hiding this comment.
We should be able to define the right incremental build pieces for whatever tooling. With api-extractor we either need to work to get the tool to do all we want in one command to ease burden on build system or have build system that is more piece agnostic. I don't have opinion on fluid-build versus others.
fluid-build seems sufficiently flexible. It seems like part of the problem is that it is smart enough that it keeps the build going for the most part even when the config is not correct. If it gave up more, then more explicit configs would be needed. Probably agents are up to the task of setting those with sufficient guidance. The debt of CSJ+ESM is still weighing on things and likely seems part of the reason agent didn't get things as good.
Without swapping to something else
- if we could define default deps for a name pattern like for "check:exports:cjs:" and "check:exports:esm:" then this would also have been easy.
- Note that the check:exports entries are mostly automated via policy checking. The checker could make sure there are build deps in place to avoid need for pattern or bunch of defaults in root config. I didn't try for that since it was reasonable to place the dep burden on the concurrency task. I think it worked right for a long while.
Sure, behind env or just try it out to see what is missing, if anything. If the error is clear an agent might be able to fix.
| const config = this.getTaskDefinition(task.taskName); | ||
| if (config !== undefined && !config.isDefault) { | ||
| task.collectNamedSubTasks(subtasksOfExplicitGroups); | ||
| } else if (config?.isDefault) { | ||
| task.collectNamedSubTasks(subtasksOfDefaultGroups); | ||
| } |
There was a problem hiding this comment.
The suggestions below are equivalent to current code and I think clearer. I don't know if any version meets the intention noted in groupTask.ts.
| const config = this.getTaskDefinition(task.taskName); | |
| if (config !== undefined && !config.isDefault) { | |
| task.collectNamedSubTasks(subtasksOfExplicitGroups); | |
| } else if (config?.isDefault) { | |
| task.collectNamedSubTasks(subtasksOfDefaultGroups); | |
| } | |
| const config = this.getTaskDefinition(task.taskName); | |
| if (config !== undefined) { | |
| if (config.isDefault) { | |
| task.collectNamedSubTasks(subtasksOfDefaultGroups); | |
| } else { | |
| task.collectNamedSubTasks(subtasksOfExplicitGroups); | |
| } | |
| } |
or even
| const config = this.getTaskDefinition(task.taskName); | |
| if (config !== undefined && !config.isDefault) { | |
| task.collectNamedSubTasks(subtasksOfExplicitGroups); | |
| } else if (config?.isDefault) { | |
| task.collectNamedSubTasks(subtasksOfDefaultGroups); | |
| } | |
| const config = this.getTaskDefinition(task.taskName); | |
| if (config !== undefined) { | |
| task.collectNamedSubTasks(config.isDefault ? subtasksOfDefaultGroups : subtasksOfExplicitGroups); | |
| } |
Summary
Named subtasks created by
concurrently(e.g.check:exports:cjs:alphafromconcurrently "npm:check:exports:*") receive the defaultafter: ["^*"]dependency independently infinalizeDependentTasks, making them depend on all scheduled tasks in upstream dependency packages. This causes unrelated changes (like touching an eslint config) to invalidate downstream api-extractorcheck:exports:*tasks.The root cause:
finalizeDependentTasksiterates all tasks including concurrently subtasks directly, bypassing theisDefaultguard inGroupTask.addDependentTasksthat was designed to prevent this.Code fix (
buildGraph.ts,task.ts,groupTask.ts)collectNamedSubTasksvirtual method toTask(no-op) /GroupTask(yields named subtasks)afterprocessing for subtasks that appear only in explicit groups — they inherit the correct dependencies from their parent viainitializeDependentLeafTasksConfig workaround (
fluidBuild.config.cjs)Adds explicit
["api"]task definitions for allcheck:exports:*subtask names so the fix takes effect immediately without waiting for the code change to ship.