Hi all. I’m running the latest version of methylseq with bismark. I’m confused why it creates multiple copies of files. It takes so much storage space. Is it expected behavior? Here is the example:

15G     2024-08-28 15:10        ./65/1381ede4324be0753367b2e15a26b1/ctrl5_1_val_1.fq.gz.temp.1_bismark_bt2_pe.bam
15G     2024-08-28 15:10        ./65/1381ede4324be0753367b2e15a26b1/ctrl5_1_val_1.fq.gz.temp.2_bismark_bt2_pe.bam
15G     2024-08-28 15:10        ./65/1381ede4324be0753367b2e15a26b1/ctrl5_1_val_1.fq.gz.temp.3_bismark_bt2_pe.bam
15G     2024-08-28 15:10        ./65/1381ede4324be0753367b2e15a26b1/ctrl5_1_val_1.fq.gz.temp.4_bismark_bt2_pe.bam
18G     2024-08-28 15:10        ./7a/43047cba0b3265ad58d30bf2d38153/ctrl4_1_val_1.fq.gz.temp.1_bismark_bt2_pe.bam
18G     2024-08-28 15:10        ./7a/43047cba0b3265ad58d30bf2d38153/ctrl4_1_val_1.fq.gz.temp.2_bismark_bt2_pe.bam
18G     2024-08-28 15:10        ./7a/43047cba0b3265ad58d30bf2d38153/ctrl4_1_val_1.fq.gz.temp.3_bismark_bt2_pe.bam
18G     2024-08-28 15:10        ./7a/43047cba0b3265ad58d30bf2d38153/ctrl4_1_val_1.fq.gz.temp.4_bismark_bt2_pe.bam

Thanks!

its true those temp files do take up lot of space but, there is a --gzip option that should bring down the size considerably. try adding that. I am considering adding it by default to the nf-core module but, it doesn't support pe fasta it seems or maybe just in

ext.args

Am I right in thinking that they're not identical copies, but chunked intermediates for faster parallel processing?

thats right

Thank you! That was my thought as well when I saw that their sizes are not identical in bytes, so I guess there are four chunks of the file due to four instances of bismark running. Good to know that I can add --gzip to reduce the sizes since the disk usage is definitely a problem