Skip to content

Fix module binaries on nextflow module run via session flag (#7087)#7089

Open
pditommaso wants to merge 1 commit intomasterfrom
draft-module-run-binaries-7087-flag
Open

Fix module binaries on nextflow module run via session flag (#7087)#7089
pditommaso wants to merge 1 commit intomasterfrom
draft-module-run-binaries-7087-flag

Conversation

@pditommaso
Copy link
Copy Markdown
Member

Summary

Fixes #7087 — alternative to #7088.

When a module is launched directly via nextflow module run <scope>/<name>, the module's main.nf is loaded as the entry script, not as an included module. As a result ScriptMeta.isModule() returns false for the script that owns the process, and TaskProcessor.getModuleBundle() returned null, so the resources/usr/bin directory was never injected into the task PATH -- even with nextflow.enable.moduleBinaries = true.

Approach

Track the "running as a module" state explicitly on the Session (run-scoped, not global), instead of widening the bundle gate:

  • CmdRun exposes a protected boolean isModuleRun() hook (default false).
  • CmdModuleRun overrides it to return true.
  • CmdRun.run() propagates the value to the runner's session: runner.session.setModuleRun(isModuleRun()).
  • TaskProcessor.getModuleBundle() now resolves the bundle when the owner script is either an included module (meta.isModule()) or the entry script of a module-run invocation (session.isModuleRun()).
// TaskProcessor.getModuleBundle()
if( meta?.scriptPath == null )
    return null
return (meta.isModule() || session.isModuleRun()) ? meta.getModuleBundle() : null

The feature remains opt-in via nextflow.enable.moduleBinaries.

How this differs from #7088

#7088 (drop the gate) This PR (session flag)
Change footprint 1 file, 1-line gate change 5 files: Session, CmdRun, CmdModuleRun, TaskProcessor, test
Gate semantics Bundle resolved whenever the owner script has a known path Bundle resolved only for included modules or module-run entry scripts
Behavior for plain nextflow run pipeline.nf with a sibling resources/ New: bundle picked up if moduleBinaries=true Unchanged: bundle still ignored (exactly preserves previous behavior)
Risk of accidental bundle pickup Low (still gated by feature flag), but the surface is broader None: the new path is only reachable via nextflow module run
Plumbing None Adds Session.moduleRun + CmdRun.isModuleRun() hook

This variation is more conservative: it fixes the reported issue without changing what counts as a module-bundle source for any existing invocation other than nextflow module run. The trade-off is more plumbing (one new flag threaded from CLI to Session).

Possible side-effects and implications

  • No effect when the feature flag is off. TaskProcessor.getBinDirs() still gates the call on session.enableModuleBinaries(), so default behavior is unchanged.
  • No effect on nextflow run (with or without include). session.isModuleRun() is false for every CLI subcommand other than nextflow module run, so the gate behaves exactly as before for those invocations.
  • No effect when there is no resources/ directory. ResourcesBundle.scan() returns an empty bundle when the directory does not exist, so getBinDirs() produces an empty list.
  • Defensive guard added. A meta?.scriptPath == null early-return preserves robustness — ScriptMeta.getModuleBundle() would otherwise throw IllegalStateException if scriptPath were not yet set.
  • API surface change. Session gains isModuleRun() / setModuleRun(boolean) and CmdRun gains a protected boolean isModuleRun() hook. These are minor additions but worth noting for downstream code that subclasses CmdRun.

Test plan

  • ./gradlew :nextflow:test --tests "nextflow.processor.TaskProcessorTest" — new parameterized test should resolve module bundle for entry script when running as module … covers all three combinations:
    • included module (isModule=true, isModuleRun=false) → bundle resolved
    • module-run entry (isModule=false, isModuleRun=true) → bundle resolved (the fix)
    • plain entry (isModule=false, isModuleRun=false) → bundle NOT resolved (unchanged)
  • ./gradlew :nextflow:test --tests "nextflow.SessionTest" --tests "nextflow.cli.CmdRunTest" passes.
  • End-to-end repro from the issue (nextflow -c test.config module run cellgeni/failexample --greeting 'Hello world!') succeeds with this change applied — recommended manual verification before merge.

When a module is launched directly via `nextflow module run`, the module
main.nf is loaded as the entry script, so `ScriptMeta.isModule()` is
false and `TaskProcessor.getModuleBundle()` was returning null, so the
`resources/usr/bin` directory was not added to the task PATH.

Track the "running as a module" state on the Session: `CmdRun` exposes a
`isModuleRun()` hook (default false) which `CmdModuleRun` overrides to
true, and `Session.moduleRun` is set from there. The bundle is now
resolved when the owner script is either an included module
(`meta.isModule()`) or the entry script of a `nextflow module run`
invocation (`session.isModuleRun()`).

The feature remains opt-in via `nextflow.enable.moduleBinaries`. A null
`scriptPath` guard is added to keep `getModuleBundle()` robust against
scripts that have not had their path registered.

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 29, 2026

Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit 9bb45b0
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/69f25803a5f4d10008e7b368
😎 Deploy Preview https://deploy-preview-7089--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@pditommaso pditommaso requested a review from bentsherman April 29, 2026 19:15
@pditommaso pditommaso changed the title [draft] Fix module binaries on nextflow module run via session flag (#7087) Fix module binaries on nextflow module run via session flag (#7087) Apr 29, 2026
@pditommaso pditommaso marked this pull request as ready for review April 29, 2026 19:16
@pditommaso
Copy link
Copy Markdown
Member Author

Likely this is cleaner option vs #7088

@pditommaso pditommaso requested a review from jorgee May 4, 2026 13:42
Copy link
Copy Markdown
Contributor

@jorgee jorgee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look fine with me, just minor comments.
I wonder if, now that modules are in production, we could create a tests namespace and include integration tests, such as the one that is failing in this issue.


Session setModuleRun(boolean value) {
this.moduleRun = value
return this
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, there is an inconsistency with setters, setBinding and this one are using fluent but others like setBaseDir and setLibDir return void. Looking at the calls looks like it is not needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Module binaries are not added to the PATH when running nextflow module run

2 participants