What is the current state of running

snakemake

inside

nextflow

, and specifically

nf-core

? There is a tool, CADD-SV, which we want to run on some of our data. I noticed that someone has implemented the caddsv get command in nf-core already, but this just pre-fetches the annotation data. The actual

caddsv run

command is a wrapper around a

snakemake

pipeline which is hardcoded to run with

--use-conda

. Is it even feasible to run such a tool within a nextflow process? Is it over burdensome with a lot of unnecessary overhead (rebuilding conda envs on every single process execution)? Is there a route to such a tool being in nf-core?

We have https://nf-co.re/modules/snakemake/ so it must be possible in some scenarios

Your other option is to use it nf-cascade style. The poc runs nextflow in nextflow, but you can use it to do snakemake in nextflow in the same way. It just means snakemake needs to be in your PATH too, but Pixi can handle that. Just let me know if you want more details

The tool let’s you set a custom conda dir for the internal snakemake envs so I am wondering if that can actually be passed into the process and mounted into any container and would work as intended?

nf-cascade

style seems like it might be quite a heavy solution given that the internal snakemake is always running in a “local” mode (it’s not setup to allow the user to distribute steps across e.g. slurm)

Ah, that's a shame, then yes, nf-cascade is overkill. For just local running, the module is optimal I think. And you can always patch it to fit your needs.

@Adam Talbot played with the snakemake modules, and if I remember well using conda, he did a full inception, nextflow > snakemake > nextflow, but I don't remember how deep he went

And how deep did you went in the inception @Adam Talbot?

You can leave it going infinitely if you don't mind breaking your computer

Next challenge: run the inception in Galaxy

Hi! I've made Galaxy wrapper for #C07Q8HNU4FK (and published it on the Toolshed by the way). The XML file is [here](github.com/OlivierCoen/stableexpression_galaxy/blob/…/nf_core_stableexpression.xml). I was able do this because nf-core/stableexpression can be run fully with conda / mamba / micromamba. In the Galaxy wrapper, I'm installing

nextflow

and

micromamba

(and

openjdk

but it may be overkill). Besides, I created a `micromamba`profile in the nf-core/stableexpression, and I'm running the pipeline inside Galaxy using this profile. So the inception here is: Galaxy (conda "engine") -> Nextflow (micromamba "engine") I've first tried with

apptainer

instead of

micromamba

, but then I had issues with fuse-fs...