Hi all, pipeline implementation question - would you expect that UTILS_NFCORE_<PIPELINENAME>_PIPELINE would ever run any processes? Or would you reserve it only for pre-processing and put anything that requires actual processing in the main pipeline workflow? I have a situation where I have an optional data type input to the pipeline, which if not provided needs to be calculated and used as part of validation for another pipeline input. So I'm torn between putting this all in one place but keeping all the validation code together, or splitting it so that UTILS_ only runs channel manipulations and the processing happens in the main codebase only.

The only thing I've changed in this subworkflow is the samplesheet validation, and then the citations paragraphs for multiqc. I would use it to add Groovy functions if I needed any more, but for pipeline logic stuff I would go straight in the pipeline subworkflows, splitting the validation logic in this case

I also think that

UTILS_

should not run processes. I think everything that runs a process should be part of the main pipeline-workflow

OK, that's two for splitting, so I'll go with that! I will instead just generate the data from the samplesheets in UTILS_, and move all the required validation for these inputs to a custom validation subworkflow.

I also wonder what that would look like in the terminal output, it would then be called

nf-PIPELINE_INITIALIZATION:PROCESS

right?

Yeah, something like that!

For what it's worth our pipeline also has a validation step that occurs after a "real" process runs a tool to extract metadata from input files. We do put as much checking as possible in the UTILS workflow but anything that requires actually reading the input files has to live in a process.