Can someone please remind me again what is the current polic nf-core #modules

Can someone please remind me again what is the cur...

Arthur

08/27/2025, 6:01 AM

Can someone please remind me again what is the current policy on containers using biocontainers vs seqera? I would like to bump the Clair3 module, but the latest versions don’t have the singularity images available via biocontainers (almost certainly they built but haven’t gone public which seems to happen a lot, so you have to make an issue and I am too lazy) so I am wondering whether it can be switched to the Seqera containers?

Phil Ewels

08/27/2025, 6:13 AM

https://nf-co.re/docs/tutorials/nf-core_components/using_seqera_containers

❤️ 1

Arthur

08/27/2025, 6:14 AM

Given that it’s going to become the default according to that I feel it’s OK to migrate then?

this 1

👍🏻 1

Sateesh_Peri

08/27/2025, 6:16 AM

yup

Maxime Garcia

08/27/2025, 7:30 AM

we started migrating some

👍 2

Arthur

08/27/2025, 7:31 AM

A little wrinkle I discovered (at least for Clair3) is that biocontainers, despite being built on top of bioconda did not put data into

CONDA_PREFIX

whilst the seqera containers do

Maxime Garcia

08/27/2025, 7:35 AM

Interesting

Arthur

08/28/2025, 4:51 AM

Ok, so I am a little perturbed because the

docker

has

CONDA_PREFIX

set, but the

singularity

does not. I thought that within seqera containers the environments should be the same but now I am not sure.

🤔 1

Phil Ewels

08/28/2025, 5:13 AM

What effect does this have? The packages are installed to a different path within the container?

Phil Ewels

08/28/2025, 5:17 AM

I don't find the string

CONDA_PREFIX

anywhere in the Wave code base

Arthur

08/28/2025, 5:19 AM

So in my specific case it’s affecting the update to Clair3. Currently it uses CONDA_PREFIX for mamba and conda profiles. However it turns out that seqera containers still put the models under those paths and not the

/usr/lib

as biocontainers did. BUT then only the docker has CONDA_PREFIX set, not the singularity image

Phil Ewels

08/28/2025, 5:23 AM

Ok 👍🏻 I'm not quite sure how that environment variable is being set, seeing as I can't find it in the codebase. But I've raised it with the Wave team (who are away on vacation currently but back next week) so we'll look into it.

Phil Ewels

08/28/2025, 5:23 AM

Might be worth making an issue on the Wave repo so it doesn't get lost / so that you get updates.

Arthur

08/28/2025, 5:52 AM

Done. I suspect this is not a

wave

issue but more likely some weird difference between docker and singularity that was either unknown or not accounted for.

👍🏻 1

Phil Ewels

08/28/2025, 8:44 AM

Thanks!

Sam Wilkinson

08/29/2025, 2:01 PM

Does anyone have any ideas what's going on with biocontainers? It seems to have completely stopped uploading new singularity containers....

James Fellows Yates

08/29/2025, 2:02 PM

You can ask on the Bioconda gitter

Sam Wilkinson

08/29/2025, 2:03 PM

Ah, I wasn't aware that such a thing existedc

Phil Ewels

08/31/2025, 8:26 PM

https://nf-co.re/blog/2024/seqera-containers-part-1#time-to-singularity-image

Sam Wilkinson

08/31/2025, 8:53 PM

Well the problem I find with seqera containers is that it takes ages for the website UI to update to allow the new version of the package

Sam Wilkinson

08/31/2025, 8:53 PM

it didn't show as available until the next morning last time I was updating a module

Phil Ewels

08/31/2025, 9:05 PM

yeah it runs nightly I think 🤔 (or maybe not, but there is some sort of cache refresh period)

Phil Ewels

08/31/2025, 9:05 PM

But that's just the web UI / cache.

Phil Ewels

08/31/2025, 9:05 PM

The underlying Wave service doesn't use that at all, so if you generate your image using the CLI then you should be able to fetch it within seconds of it appearing on Bioconda

Phil Ewels

08/31/2025, 9:07 PM

Copy code

wave --freeze --conda multiqc

For example..

Phil Ewels

08/31/2025, 9:07 PM

Wave CLI installation instructions / source code are here

Phil Ewels

08/31/2025, 9:10 PM

Getting the ORAS singularity image is similar. Getting the HTTPS singularity image is a little more complicated, needs some finagling with

wave inspect

but I wrapped that logic in a Python snippet here.

Phil Ewels

08/31/2025, 9:10 PM

Intention is to fully automate all of this gubbins in nf-core modules as soon as possible. We are making progress in that direction, slowly.. (very slowly)

Phil Ewels

08/31/2025, 9:12 PM

We use our own custom cache of package names on the Seqera Containers web UI because the underlying Anaconda / PyPI API endpoints are pretty terrible and extremely slow. So either you wait a bit for new packages to appear, or you wait an annoying number of seconds for every search. I figure that most of the time, people don't care about bleeding edge releases, so that was the lesser of two evils.

Sam Wilkinson

09/01/2025, 9:22 AM

I looked at using wave but when I saw it needed a token I stopped, I assume it's free and open like the web UI?

Phil Ewels

09/01/2025, 9:22 AM

Yup 👍🏻

Phil Ewels

09/01/2025, 9:22 AM

If you use

--freeze

and no registry then it pushes to the open community registry

Phil Ewels

09/01/2025, 9:23 AM

It's exactly what the web UI does, the web UI just provides a token on the back end for a bot user to run the build

Sam Wilkinson

09/01/2025, 9:23 AM

caching the indexes definitely makes sense, responsiveness is definitely better, suggest maybe having a little note somewhere saying when the last cache happened though

💡 1

Phil Ewels

09/01/2025, 9:23 AM

But the builds are the same and the location is the same

👍 1

Open in Slack

Previous Next