So I had a passing thought. What happens with the workdir when using meta pipelines? • Do we get a working dir per sub-pipeline? • Does everything go into one huge directory? • Can we selectively delete the workdir for each sub pipeline while keeping resumability? I can see this go very wrong very fast on some HPC clusters

Meta pipelines will basically function as any other pipeline so there's only one work directory

maybe we can give it a (nf-)boost?

good point 😅

That's on them to not put 1,0000,00000000 samples in one run 😉

single samples can get pretty big nowadays 😄 if you start staging annotation databases and references on the fly it can get out of hand very quick

I do think that addressing the footprint of workflows is a concern we should think about, and i've been arguing for something nf-boost like natively for a while, but it's true that meta pipelines make it even more urgent.

give it a try: github.com/nextflow-io/nextflow/pull/3849

yass