Also; some follow ups would be if it's possible to deal with proprietary/non-free software that way (in practice, I can come up with a scheme to have a conda channel provide that, but it might be more work when it's worth).

Also, does anyone know what

wave.freeze

really does, or rather if there's any risk of it being "undone" so that wave would trigger a new build for a

environment.yml

it has previously built a container for? I'm thinking about whatever wave generated containers can be used in a way that allows them to be considered tested and verified.

Also, offline support will probably need to be a thing, so there I'm wondering if anyone knows if downloading a pipeline (with container images) with wave is supported at the moment.

we no longer have an

arm.config

or similar with alternate container definitions (new URLs) but rather rely on wave and have the strategy prefer conda builds.

No, this is incorrect. We will use Wave to build 2x containers for every package: one arm, one x86. We also keep the conda lockfiles for reproducible conda environments to avoid drift (the old conda environment.yml is still there but you have to ask for it specifically in a new profile name)

proprietary/non-free software

I'm not really clear on how the containerisation or arm64 fits into this? For nf-core if there is software like this that is needed, we usually say you need to BYO container / conda environment.

>

wave.freeze

Basically this tells Wave to push the container to the specified OCI registry when the build is complete (if none specified, that is the Seqera Containers registry: commuity.wave.seqera.io ) Wave knows the hash of the request that was made for the container, so if another request comes in with the same hash it'll then just redirect directly to the previously built image rather than rebuilding New builds can be triggered if the image is deleted from the registry (Seqera Containers public registry never does this, we commit to keeping images for at least 5 years), or if something causes the cache key to change. For us the main thing that makes this likely is if the build Dockerfile changes. When you request a build with conda packages it uses a template Dockerfile. For nf-core I think we are supplying our own now (?) but also the default can change if we want to update or change something (eg. update miniconda, use pixi instead, whatever). Doing this then invalidates the hash for future requests for the same conda packages.

offline support

Wave is fundamentally a remote service, so if you want to request an image from conda packages you need to be able to make a connection to Wave. This is one of the main drivers for nf-core to use Wave only on the automated build side and not just bundle it directly into nf-core pipelines (which would be a hell of a lot simpler 😆 ). So for nf-core, when a new module turns up we request images from Wave / Seqera Containers in the CI. We then take the resulting docker URIs and save those into the config files. This means that the pipelines get big config files directly listing the container URIs without wave, which allows all of the usual "download for offline use" machinery to work as usual - Nextflow doesn't need to work out the container URI at run time, which means that it can check a local path for the image.

I'm thinking about whatever wave generated containers can be used in a way that allows them to be considered tested and verified.

See above. Wave is dynamic by design (that's kind of its main feature), so if you want them to be static for "verified" status of some kind, you need to use it only at build time and not run time.

Note that Wave does have some nice features like security scanning (stop Nextflow from running if containers have known CVEs) and other stuff that obviously only works if you use Wave at runtime.

The Wave documentation has historically been pretty awful, but we've been working on them a lot recently so worth checking out. There are a bunch of technical mermaid diagrams now explaining how stuff like freeze works under the hood if you're curious: docs.seqera.io/wave/features/container-freezes#…

Regarding slack channels, for nf-core specifically we have the #C09V4RAHV0X channel for discussion around the effort to migrate away from BioContainers to Seqera Containers

On the Nextflow Slack we have a

#wave-containers

channel for all things Wave

And on community.seqera.io we have a tag for

#wave

that flags posts to our internal Seqera slack Wave team 🤫 wave containers

Hope that helps!

The

arm64

(in tools templates) profile sets

wave.strategy  = 'conda,container'

(along with

wave.enabled

et.c.) which I would have thought would mean it preferred requesting a image from wave rather than use the one from the

container

directive. Is that profile obsolete and something that's picked up automatically somehow?

To me, it's clearly best from most viewpoints to have wave as you describe it above (as a image maker and registry), but it does seem to make mixed environments more difficult to handle.

I'm not sure how up to date it is, but this is the GitHub Project that we're using to track progress on the migration: github.com/orgs/nf-core/projects/90

Note that the latest release of #CE5LG7WMB included new functionality for creating containers using Wave and doing all of the nf-core specific config stuff, which is a fairly major milestone in this project. See nf-co.re/blog/2026/tools-4_1_0#…

Actually porting (bio)conda packages to have arm support was also a massive project, which has been mostly done now. See nf-co.re/blog/2026/arm-pipelines

• Modules build time ◦ On change, CI will run #CE5LG7WMB to fetch new images and record their URIs in

meta.yml

• Pipeline dev time ◦ On pulling in new modules, #CE5LG7WMB will record URIs in auto-generated pipeline config files • Pipeline run time ◦ On run, configs will point to containers for each process, depending on the profile used (CPU arch + container enginer)

(

process.arch

)

I'm not sure how even Nextflow would know what it's submitting to if it's that dynamic? 🤔

(if wave were used,

process.arch

would need to be dealt with to provide the right answer, but it doesn't seem like it will help here.)

To be truly dynamic it'd need to supply both so that when the task starts it can auto-select or something?

(You can have different "data" blocks for e.g. file systems, but it's not all there.) And I'm not sure if

oras

can differ based on the architecture used when fetching. That would be confusing at least, so not something I'd want.

I'll need to think for a bit, but I have the feeling supporting these "properly" may be difficult, especially given the (good) changes with making config strict which I think highly limits the amount of introspection that can be done. But probably it might be good enough if there would be a tool to go through a pipeline and give a good guess of what processes would be GPU enabled (have

accelerator

set to something other than 0/null) and write out a config that the user can run with and point the containers for those processes to the correct ones for their respective architectures.

I would love to have config syntax in Nextflow that allows specifying multiple

container

strings that are selected according to what

arch

is. Then we would only need to toggle

process.arch

and the rest of the config would sort itself out. Pretty sure I've suggested that before years ago and got no-where though. I might take another poke at it.

That wouldn't be sufficient for you here as accelerator != arch, but at least it'd make it simpler

But yes, having a map for

container

would be nice.

Until then I think that the above doesn't really help us all that much, if we need infra for Singularity then we may as well make it consistent for both I thinik.

Was just thinking about this..

process.container

can already take a deferred gstring, right? I mean the old style we use already has a conditional:

container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
        '<https://depot.galaxyproject.org/singularity/ubuntu:20.04>' :
        'nf-core/ubuntu:20.04' }"

Could you not use the same trick here?

I forget now why we're not doing this in #C09V4RAHV0X 🤔 Pretty sure we discussed it (like, 2 years ago)

Also note that

process.arch

can now take multiple arguments:

process.arch = 'linux/amd64,linux/arm64'

See github.com/nextflow-io/nextflow/pull/6897

As for

process.arch

allowing for multi-values, I don't think I see the benefit of that for now (but that doesn't mean there isn't one).

I think it was primarily introduced for Intelligent Compute, though of course it could also be used for other things.

(intelligent compute is executor agnostic to a degree, but that means we can add a handful of popular clouds, not that we can add a custom HPC setup for everyone - so will remain cloud only as far as I know)