Hello all, I am having a nextflow pipeline designed to do the bacteria WGS analysis. It works well. It is designed with modulization in mind. For example, the workflow calls a list of subworkflows (e.g. qc.nf, assembly.nf, annotation.nf. ), modules. Because some of the subworkflow can also be used in other pipelines. We are now thinking of breaking the subworkflows into individual workflow pipeline. Then we call these small pipelines to build the other big pipeline. The reason we want to do this is for better maintenance. For example, when we change sequence qc step, we only need to change the qc pipeline and all the pipelines calling the qc pipeline will be automatically updated. I am wondering what is the best way to do it?

Welcome @Xiaoli Dong! No need to double-post, so I've deleted the duplicate question from #CE6SDEDAA - hope that's ok.

Thank you @Phil Ewels. Here is the pipeline link. https://github.com/xiaoli-dong/pathogenseq Now, we want to make some of the subworkflow into independent pipelines. for example, qc_illumina pipeline, qc_nanopore pipeline. Those pipelines can later on be called by the other pipelines....

So chopping up pipelines is easy, the tricky bit is glueing them back together again. This is typically referred to as "pipeline chaining". There aren't really any great solutions for this currently, though it's a known problem so I'm hopeful that we'll be able to address it in the near- to mid-future.

Are you aware of Bactopia? It is not an nf-core pipeline but is very mature and well-supported AFAIK.

yes