Hi I have been running rnavar on 480 samples and i...
# rnavar
y
Hi I have been running rnavar on 480 samples and it doesn't work after couple of sample ran bc all the files saved in work folder use more quota than I have so my job get halted Has anyone had same issues ? Do you remove everything in work in between the batches of samples ?
n
Hi this is a common issue with nextflow pipelines. The developers of Nextflow are planning to add dynamic cleanup of the work directory some time in the future, but in the meantime, you can try to use the cleanup option of nf-boost: https://github.com/bentsherman/nf-boost?tab=readme-ov-file#configuration
Keep in mind that this is an experimental feature and that it currently breaks the resumeability of your run
party this 1
t
This is a fairly common problem across many pipelines. Have a look at this discussion from #CE8SSJV3N for some other options. With rnaseq, batches work well, but this depends on whether any of the analyses need all samples to be available at the same time, like joint variant calling. If not, the results should be the same.