Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions docs/container.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,88 @@ Nextflow supports a variety of container runtimes. Containerization allows you t
When creating a container image to use with Nextflow, make sure that Bash (3.x or later) and `ps` are installed in the image, along with other tools required for collecting metrics (See {ref}`this section <execution-report-tasks>`). Bash should be available on the path `/bin/bash` and it should be the container entrypoint.
:::

(container-apple)=

## Apple container

[Apple container](https://github.com/apple/container) is a lightweight container runtime that runs each container inside its own virtual machine on macOS, using the [Virtualization framework](https://developer.apple.com/documentation/virtualization).

### Prerequisites

You will need [Apple container](https://github.com/apple/container) installed on your macOS system. It requires Apple silicon (M1 or newer) and a recent macOS release.

Start the system service before running your pipeline:

```bash
container system start
```

### How it works

You won't need to modify your Nextflow script in order to run it with Apple container. Simply enable it in the Nextflow configuration file:

```groovy
process.container = 'nextflow/examples:latest'
appleContainer.enabled = true
```

Every time your script launches a process execution, Nextflow will run it inside a container created from the specified image. In practice Nextflow will automatically wrap your processes and run them by executing the `container run` command with the image you have provided.

:::{note}
Since each container runs in its own lightweight VM, tasks benefit from strong isolation without the overhead of a full VM per task. Images are standard OCI images and can be pulled from any registry.
:::

:::{note}
Apple container only supports Linux images. When running an `amd64` image on Apple silicon, pass `--rosetta` via `appleContainer.runOptions` and set the platform explicitly, e.g. `process.arch = 'linux/amd64'`.
:::

### Multiple containers

It is possible to specify a different image for each process definition in your pipeline script. Suppose you have two processes named `hello` and `bye`. You can specify two different images for them in the Nextflow script as shown below:

```nextflow
process hello {
container 'image_name_1'

script:
"""
do this
"""
}

process bye {
container 'image_name_2'

script:
"""
do that
"""
}
```

Alternatively, the same container definitions can be provided by using the configuration file as shown below:

```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}

appleContainer {
enabled = true
}
```

Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.

### Advanced settings

Apple container advanced configuration settings are described in {ref}`config-apple-container` section in the Nextflow configuration page.

(container-apptainer)=

## Apptainer
Expand Down
38 changes: 38 additions & 0 deletions docs/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,44 @@ The following settings are available:
`workDir`
: The pipeline work directory. Equivalent to the `-work-dir` option of the `run` command.

(config-apple-container)=

## `appleContainer`

The `appleContainer` scope controls how [Apple container](https://github.com/apple/container) is used by Nextflow.

The following settings are available:

`appleContainer.enabled`
: Execute tasks with Apple container (default: `false`).

`appleContainer.engineOptions`
: Specify additional options supported by the `container` CLI i.e. `container [OPTIONS] run`.

`appleContainer.envWhitelist`
: Comma separated list of environment variable names to be included in the container environment.

`appleContainer.kill`
: Set the signal used to stop a running container (default: `true`, which sends `SIGTERM` via `container stop`). Set to a signal name (e.g. `'SIGKILL'`) to use `container kill -s`, or `false` to disable.

`appleContainer.registry`
: The registry from where container images are pulled. It should NOT include the protocol prefix i.e. `http://`.

`appleContainer.remove`
: Clean up the container after the execution (default: `true`).

`appleContainer.runOptions`
: Specify extra command line options supported by the `container run` command (e.g. `--rosetta`, `--ssh`).

`appleContainer.temp`
: Mounts a path of your choice as the `/tmp` directory in the container.

`appleContainer.tty`
: Allocate a pseudo-tty (default: `false`).

`appleContainer.writableInputMounts`
: When `false`, mount input directories as read-only (default: `true`).

(config-apptainer)=

## `apptainer`
Expand Down
2 changes: 2 additions & 0 deletions modules/nextflow/src/main/groovy/nextflow/Session.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import nextflow.cache.CacheDB
import nextflow.cache.CacheFactory
import nextflow.conda.CondaConfig
import nextflow.config.Manifest
import nextflow.container.AppleContainerConfig
import nextflow.container.ApptainerConfig
import nextflow.container.CharliecloudConfig
import nextflow.container.ContainerConfig
Expand Down Expand Up @@ -1187,6 +1188,7 @@ class Session implements ISession {
new SingularityConfig(config.singularity as Map ?: Collections.emptyMap()),
new ApptainerConfig(config.apptainer as Map ?: Collections.emptyMap()),
new CharliecloudConfig(config.charliecloud as Map ?: Collections.emptyMap()),
new AppleContainerConfig(config.appleContainer as Map ?: Collections.emptyMap()),
] as List<ContainerConfig>

if( engine ) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
/*
* Copyright 2013-2026, Seqera Labs
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package nextflow.container

import groovy.transform.CompileStatic
import groovy.util.logging.Slf4j

/**
* Wrap a task execution inside an Apple container (apple/container) runtime.
*
* @author Paolo Di Tommaso <paolo.ditommaso@gmail.com>
*/
@Slf4j
@CompileStatic
class AppleContainerBuilder extends ContainerBuilder<AppleContainerBuilder> {

private boolean remove

private boolean tty

private String name

private String capAdd

private String removeCommand

private String killCommand

private kill = true

AppleContainerBuilder(String name, AppleContainerConfig config) {
this.image = name

if( config.engineOptions )
addEngineOptions(config.engineOptions)

if( config.runOptions )
addRunOptions(config.runOptions)

if( config.temp )
this.temp = config.temp

this.remove = config.remove
this.tty = config.tty

if( !config.writableInputMounts )
this.readOnlyInputs = true
}

AppleContainerBuilder(String name) {
this(name, new AppleContainerConfig([:]))
}

@Override
AppleContainerBuilder params( Map params ) {
if( !params ) return this

if( params.containsKey('entry') )
this.entryPoint = params.entry

if( params.containsKey('kill') )
this.kill = params.kill

if( params.containsKey('capAdd') )
this.capAdd = params.capAdd

return this
}

@Override
AppleContainerBuilder setName( String name ) {
this.name = name
return this
}

@Override
AppleContainerBuilder build(StringBuilder result) {
assert image

result << 'container '

if( engineOptions )
result << engineOptions.join(' ') << ' '

result << 'run -i '

if( tty )
result << '-t '

if( cpus )
result << "--cpus ${cpus} "

if( memory )
result << "-m ${memory} "

if( platform )
result << "--platform ${platform} "

// environment variables
appendEnv(result)

if( temp )
result << "-v $temp:/tmp "

// volume mounts
result << makeVolumes(mounts)
result << '-w "$NXF_TASK_WORKDIR" '

if( entryPoint )
result << '--entrypoint ' << entryPoint << ' '

if( runOptions )
result << runOptions.join(' ') << ' '

if( capAdd )
result << '--cap-add ' << capAdd << ' '

if( name )
result << '--name ' << name << ' '

// image is the final positional argument
result << image

runCommand = result.toString()

if( remove && name ) {
removeCommand = 'container rm ' + name
}

if( kill ) {
killCommand = 'container stop '
if( kill instanceof String ) killCommand = "container kill -s $kill "
killCommand += name
}

return this
}

@Override
String getRemoveCommand() { removeCommand }

@Override
String getKillCommand() { killCommand }
}
Loading
Loading