Skip to content

Add WorkflowInterceptor extension point for named-workflow plugins#7076

Open
huangzhibo wants to merge 2 commits intonextflow-io:masterfrom
huangzhibo:add-workflow-interceptor
Open

Add WorkflowInterceptor extension point for named-workflow plugins#7076
huangzhibo wants to merge 2 commits intonextflow-io:masterfrom
huangzhibo:add-workflow-interceptor

Conversation

@huangzhibo
Copy link
Copy Markdown

@huangzhibo huangzhibo commented Apr 24, 2026

Context

Relates to #5589 (checkpoint / skip-forward discussion).

Adds a minimal pf4j ExtensionPointnextflow.plugin.WorkflowInterceptor — that lets a plugin wrap the execution of named workflows. The core change is tiny; the actual stage-archiving / reuse logic lives in a separate plugin, nf-stage, which is the first consumer.

What changes

  • New interface WorkflowInterceptor in nf-commons (32 lines)
  • WorkflowDef.run() looks up an interceptor via Plugins.getExtension(WorkflowInterceptor); if one is registered, delegates to it with a proceed closure that runs the original body (13 lines, extracted into runDefault())
  • Entry workflows (name == null) are never intercepted
  • Behavior is unchanged when no plugin is installed

Total: +45 / -0 across 2 files.

API

interface WorkflowInterceptor extends ExtensionPoint {
    Object intercept(Object workflow, Object[] args, Closure proceed)
}

The interceptor owns the invocation: it may mutate args (e.g. clone channel inputs in place) before calling proceed(), and post-process the returned ChannelOut. Whether and when proceed() is invoked, and how the result channels are driven, is the plugin's responsibility.

Why an extension point

Issue #5589 surfaces two positions: users want stage-level skip-forward semantics; core maintainers want -resume left alone and prefer pipeline authors prototype the pattern first. An extension point sidesteps the tension — it doesn't add any user-facing semantics to core, but unblocks plugin-side experimentation. If a pattern proves out in a plugin, promoting it to core remains an option later.

Consumer: nf-stage

nf-stage uses this hook to archive each named workflow's emit: outputs on first run and, on reruns, replay them from a content-addressed archive when a SHA-256 digest over (stage name, static args, channel input contents) matches — no work/ dependency, no per-task hash. The mechanics (clone channel args into args[], let proceed() wire processes onto the clones, decide after digest compute whether to feed clones or STOP them) rely on the interceptor controlling args mutation and post-proceed channel flow, which is why the hook signature includes args[] and a proceed closure rather than simple before/after callbacks.

Test plan

  • Existing WorkflowDef tests pass (no behavior change without a plugin)
  • Validated in nf-stage end-to-end: named sub-workflows intercepted, entry workflow not intercepted, proceed() round-trips ChannelOut correctly
  • Happy to add a core-side test with a mock interceptor if reviewers prefer

Introduce a pf4j ExtensionPoint that allows plugins to intercept
named workflow execution. This enables stage-level archiving and
resume capabilities without modifying workflow scripts.

- Add WorkflowInterceptor interface in nf-commons (ExtensionPoint)
- Extract WorkflowDef.runDefault() from run() (pure refactoring)
- Add interceptor lookup via Plugins.getExtension() in run()
- Entry workflows (name is null) are never intercepted

Signed-off-by: Zhibo Huang <huangzhibo@gmail.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: huangzhibo <zhibo90@126.com>
@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 24, 2026

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit 8476068
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/69f1fc0608c0b5000896c411

@huangzhibo huangzhibo changed the title Add WorkflowInterceptor extension point for plugin-based workflow interception Add WorkflowInterceptor extension point for named-workflow plugins Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant