Skip to content

Fix module binaries PATH not injected on nextflow module run (#7087)#7088

Open
pditommaso wants to merge 1 commit intomasterfrom
fix-module-run-binaries-7087
Open

Fix module binaries PATH not injected on nextflow module run (#7087)#7088
pditommaso wants to merge 1 commit intomasterfrom
fix-module-run-binaries-7087

Conversation

@pditommaso
Copy link
Copy Markdown
Member

Summary

Fixes #7087.

When a module is launched directly via nextflow module run <scope>/<name>, the module's main.nf is loaded as the entry script, not as an included module. As a result ScriptMeta.isModule() returns false for the script that owns the process, and TaskProcessor.getModuleBundle() returned null, so the resources/usr/bin (and resources/bin, resources/usr/local/bin) directory was never injected into the task PATH -- even with nextflow.enable.moduleBinaries = true.

This was inconsistent with nextflow run + include { … } from './modules/...', which marks the included script as a module and gets the PATH injection correctly.

Change

Single-line change in TaskProcessor.getModuleBundle() -- switch the gate from isModule() to getScriptPath():

return meta?.getScriptPath() ? meta.getModuleBundle() : null

The bundle is now resolved whenever the owner script has a known path, regardless of whether the script was loaded as the main entry or as an included module. The feature remains opt-in via nextflow.enable.moduleBinaries, and ResourcesBundle.scan() still returns an empty bundle when no resources/ directory exists, so this is a no-op for scripts without one.

Why this approach

I considered three options (described in the linked issue investigation):

  1. Drop the isModule() gate -- this PR. Smallest, most direct fix.
  2. Mark the entry script as a module via setModule(true) -- not viable: in ScriptLoaderV2 setModule(true) also flips skipEntryFlow=true, which would prevent the implicit-workflow execution that nextflow module run relies on. Would require a new flag (e.g. isEntryModule) -- more invasive.
  3. Have CmdModuleRun synthesize a wrapper script that includes the module -- larger refactor.

Option 1 was chosen as the minimal and semantically correct fix.

Possible side-effects and implications

The change widens the set of scripts that are eligible to contribute a resources/ bundle to the task environment. Concretely:

  • Behavior change (intended). A standalone pipeline script pipeline.nf that has a sibling resources/usr/bin/ (or resources/bin/, resources/usr/local/bin/) directory will now also get those directories prepended to the task PATH, only when the user explicitly opts in with nextflow.enable.moduleBinaries = true. Previously this was silently ignored unless the script was loaded via include. This is arguably the correct semantics of the feature flag and matches how an end user would expect it to work.

  • No effect when the feature flag is off. TaskProcessor.getBinDirs() still gates the call on session.enableModuleBinaries(), so behaviour is unchanged for the default configuration.

  • No effect when there is no resources/ directory. ResourcesBundle.scan() returns an empty bundle when the directory does not exist (bundle/ResourcesBundle.groovy:147-148), so getBinDirs() produces an empty list and PATH is not modified.

  • No effect on the include-based path. Scripts loaded as included modules continue to have getScriptPath() set, so they keep working exactly as before.

  • Bundle staging / fingerprinting. When the bundle is non-empty, ResourcesBundle.fingerprint() is part of cache-key inputs in some code paths (e.g. wave/container-bundle scenarios). For a previously "non-module" script that now has a sibling resources/, the fingerprint of cached tasks could change only the first time the user opts into moduleBinaries. Since opting in is an explicit user action this is acceptable; existing module-run / include-based pipelines are unaffected.

  • Defensive guard kept. The check now uses getScriptPath() rather than dropping the guard entirely, because ScriptMeta.getModuleBundle() throws IllegalStateException when scriptPath is null. This preserves robustness for any code path that happens to register a script without setting its path.

Test plan

  • ./gradlew :nextflow:test --tests "nextflow.processor.TaskProcessorTest" passes (includes a new test case should resolve module bundle when script path is set regardless of isModule that exercises the fix).
  • ./gradlew :nextflow:test --tests "nextflow.script.ScriptMetaTest" --tests "nextflow.SessionTest" --tests "nextflow.cli.CmdRunTest" -- existing tests for related areas still pass.
  • End-to-end repro from the issue (nextflow -c test.config module run cellgeni/failexample --greeting 'Hello world!') succeeds with this change applied -- recommended manual verification before merge.

When a module is launched directly via `nextflow module run`, the module
main.nf is loaded as the entry script, so `ScriptMeta.isModule()` is
false and `TaskProcessor.getModuleBundle()` was returning null, causing
the `resources/usr/bin` directory not to be added to the task PATH.

Switch the gate from `isModule()` to `getScriptPath()` so the bundle is
resolved whenever the owner script has a known path. This makes the
behavior of `nextflow module run` consistent with `nextflow run` plus
include. The feature remains opt-in via `nextflow.enable.moduleBinaries`.

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 29, 2026

Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit 196e631
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/69f25455c935e50008768292
😎 Deploy Preview https://deploy-preview-7088--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@pditommaso pditommaso requested a review from bentsherman April 29, 2026 18:57
@pditommaso
Copy link
Copy Markdown
Member Author

@bentsherman what do you think about this? it may be worth adding a specific flag e.g. moduleEntry ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Module binaries are not added to the PATH when running nextflow module run

1 participant