We can't use conda lockfiles as I suggested, because that would change the hash of the Wave-generated image so the Singularity Containers URI would be different. It's a catch-22 with the lockfile - Wave generates them but they can't be used to regenerate exactly the same container.

However, I went back to the discussion (pre-summer holiday 😅) and remembered that although we said no to storing the container URIs at module level in a Nextflow config, we said that we would store them at module level in the

meta.yml

file

This means that they can be locked at module level.

nf-core modules install

then simply reads the YAML files and write a Nextflow config file, without needing to run

nextflow inspect

or contact the Wave API or anything. It's all just flat files.

This means that anyone using a given shared nf-core/module will always get exactly the same images.

eg.

-profile conda

or

-profile conda_arm64

would both

includeConfig

a file full of process specifications pointing to conda lock files. Same setup as the config files for docker / singularity files.

(Side note: means we'll have 6 of these files, 8 if we also support osx/arm64 in the future, so we should probably put them into a subfolder under

config

)

Hi both - don't suppose you have a favourite command / method for generating lock files that will work for us that's handy?

Something that we can add into the Wave Dockerfile to generate the lock file.

'cc also @Mahesh Binzer-Panchal

I'm also struggling a little to understand the finer points of the conversation here. What was the original aim?

We were talking about having exactly reproducible containers across multiple pipelines, but I had forgotten that we had already solved this issue a different way.

I was also just talking to Paolo about Pixi, as he has a draft PR for using that in Wave instead of micromamba. But I'm a little wary of using Pixi lockfiles, as I don't think that they can be used with vanilla conda.

My thought is that we can use the conda lock files for all conda users, to get precise builds.

https://pixi.sh/latest/reference/pixi_configuration/#detached-environments Is the thing I'm referring to. Seems new

This thread is quite interesting, sounds like it's being discussed actively

So this is what I'm looking for I think: https://github.com/synapticarbors/pixi2ces though I would prefer it to be part of Pixi itself / more officially adopted.

Ideally Wave could then generate conda lock files and Pixi lock files, and people could use whichever they want.

We may need a

pixi

namespace in Nextflow to be able to use those lockfiles though 🤔