Hi folks! I have a suggestion / proposal.. Currently, modules don't specify a docker container URI, to make it easy to use a custom registry mirror. Nextflow defaults to docker hub, and we override this default in the pipeline template config file, setting it to quay.io This is fine until you try to run the module in isolation. Now you have no pipeline config, so Nextflow tries to use dockerhub and fails. Instead, could we go back to having the full URI in the module, and tell people to use the relatively new

registryOverride

(nextflow 25.10), making the module truly self-contained? https://docs.seqera.io/nextflow/reference/config#dockerregistryoverride This will require a mega-modules PR to update all modules in sync with a new tools / template update, but shouldn't be too disruptive I think. If a full URI is given I'm pretty sure the registry config option is ignore (though we should check this). And I think we're pushing the minimum Nextflow version to 25.10 anyway in the next template release (right, @mashehu ?). The only people who'd need to change anything are the folks swapping the registry for their mirror, and the new config option should work in a backwards-compatible manner with pipelines that aren't updated yet, I think.

Did someone say mega-PR? 👀

shared a GIF

why not do it together with container conversion?

Agreeing with @mashehu

I think it might become a big thing with the next nf release

in the meantime for people running modules from nf-core in isolation, could we point them to a custom config file with some the whatever.registry for quay.io?

we could fix my versions thing at the same time! Agree with the OP

what's your versions thing?

That would take some more work though, let's keep those things separate please

Yeah the containers should hopefully be a pretty trivial change, I'm hoping.. 🤞🏻

a-little-girl-in-a-black-dress-and-white-collar-is-making-a-funny-face-and-says-famous-last-words.gif

I came to it because of the upcoming

nextflow modules run nf-core/xxx

command. At the moment this, fails as it needs docker (or something):

nextflow modules run nf-core/fastqc --meta.id sample1 --input sample1.fq.gz

I want to get it to work by adding

-with-docker

- that fails right now with a validation error, but hoping to get around that. But then it still fails because the

container

is not valid in isolation.

Make modules modular again!

I think that we can actually update modules and tools separately.. Pretty sure that

docker.registry

will just ignore all fully qualified URLs?

This:

Instead, could we go back to having the full URI in the module, and tell people to use the relatively new

registryOverride

(nextflow 25.10), making the module truly self-contained?

Is exactly what I was asking for in the first place! the docker.registry thing was a hack. I actually think this should be solved with a pull-through cache or Wave but that's a different matter.

Yup

Screenshot 2026-04-21 at 11.32.24.png

But not very many lines changed 😅

I hope all those CI tests pass eyebrows

uh, wot?

Too many concurrent tests?

256 sad cowboy

that's why we can't do any bulk things, but split them up

shared a GIF

YOLO?

shared a GIF

Feels like a great use case for that agentic stuff: • take a list of modules that need updating • open a PR for each of them • if tests pass with no changes, merge • if they don't, make a low effort attempt to fix • if they still don't, ask a human

shared a GIF

nf-claw anyone

It'll never catch on

image.png

Phil desperate to get up the nf-core/modules leaderboard

Basically all of my PRs to modules are these bulk ones 😆

Someone should check that

nf-core pipelines download

still works. It does a bit of parsing of the container names to create symlinks for the same container in other registries. It should support fully-qualified URIs but 😨

I think it now relies completely on

nextflow inspect

which should give the right names

If we want to do this, we can bulk merge without the CI. Splitting into batches will just highlight a bunch of unrelated errors anyway and kill polar bears

Posted this yesterday, so hoping to merge today if everyone is happy.

Yaeh, I saw the PR, just responding to the suggestions to split it up

ok shall we merge the badger then? big red rubber duck nyancat

We could also add apptainer support 🤔 : https://nfcore.slack.com/archives/C043UU89KKQ/p1762852100367839?thread_ts=1762446145.074379&cid=C043UU89KKQ

Yeah, while we're merging big changes, let's add in that

The new Seqera Container upgrade are going to fix that though 🧌 😆

mmhmm

haha, but yeah sure that's fine by me

but when

Sorry, the containers thing was just me trolling @Simon Pearce, not serious

I'm just excited about the new container goodness when it happens

I think it makes sense to make this change too

I think let's do that as a separate mega-PR. We can just merge one after another.

We should be getting rid of task.ext.singularity_pull_docker_container as well

🤣

I won't tell him

It used to be a real problem back in the day, the .sif files randomly not working a lot of the time

If I remember well the issue was not the sif files themselves, but the fact that the singularity pull was not working directly from the galaxy servers when used in certain settings (ie the Crick)

You have a better memory than me, I have no idea

pff, you would anyway just blame a certain drawing by kurt vonnegut

For me this registry was mainly driven by the scidev, so if Jon says it's ok, and cause no issues with customers/users, and we can still keep equivalent registry changes (basically the whole mirroring of registry), I'm happy

I never liked it, caused me more problems that it solved. Customers won't get hit unless we do something really stupid in workflow module updates. Approved!

Done merged

looks like linting can't handle the full path. will make a quick fix

Are we ready for https://github.com/nf-core/modules/pull/11260/ to be force merged?

If @mashehu is happy (we were just chatting about it, I think so?) then I think so

gave my final ✅ already https://github.com/nf-core/modules/pull/11260/#pullrequestreview-4160960438

shared a GIF

phil can you rerun your "script" for both PRs again. i guess some PRs slipped through

(as 180 of them predate these)

jimmy-fallon-twins-gif-by-the-tonight-show-starring-jimmy-fallon.gif

shame there's no way to do this for all open PRs 🤔

the registry PR is now part of linting, so new PRs will fail there

Which one?

the one that broke everything yesterday 😉 https://github.com/nf-core/modules/pull/11251

oh registry as in quay.io registry, gotcha

nobody is talking about that one 😜

I might see if I can get Claude to re-trigger all

nf-core lint modules

CI jobs on all open PRs..

no need to run lint imo (will generate too many other errors). would just have a quick grep for missing quay.io entries

yeah but that doesn't work for open PRs

Did it merge in the most recent changes first?

Thank you!

Most likely I broke it when I merged the branch into your PR