In the script section, is it nf-core compliant to ...
# nostupidquestions
g
In the script section, is it nf-core compliant to cp/mv/rm files in the script section of a module? According to this (https://www.nextflow.io/docs/latest/working-with-files.html#copying-files), it is not but there is a dualrnaseq pipeline that I'm using that copies files using cp (https://github.com/nf-core/dualrnaseq/blob/1.0.0/main.nf).
r
generally not ideal and should be avoided. the dualrnaseq hasn't been updated in 4 years, maybe look at more recently maintained pipelines for patterns
party this 1
g
I understand that the any files that are manipulated should mainly go through the input and output but is there a specific reason/s why that is not encouraged? Maybe some documentation that explains why?
p
Because it duplicates disk space usage in the work directories, and there's almost always no need for it.
Nextflow is able to pull out specific files and dynamically name them / locate them however you like in the downstream task. So there's very rarely a need to mv or cp them.
Note that on blob storage like s3, there is no way to move a file. So doing a mv operation effectively just copies it, with the speed + expense penalties that come with that.
g
Thank you so much!!