Hej all, I am having problems being carried out of...
# rnasplice
s
Hej all, I am having problems being carried out of space quickly when running the pipeline for a big dataset (195 samples) at Uppmax. I reached 7 Tb on the work directory at the very beginning of the pipeline when running STAR for the first samples. Is there anything that can be done? Is there one that can reduce storage needs somehow (like e.g. sarek). Thanks! =)
Btw, I already have the max storage quota (10TB) for a Small Project at Uppmax, so to ask for more space is not an option and I have no idea how to circumvent the huge work directory to be able to finish running the pipeline. 😐
p
Can you run fewer samples at a time?
on-the-fly
work
directory removal is coming, but it's not ready quite yet..
s
I will try to do that. Thanks a bunch!
p
Sometimes the simple answers are the best πŸ˜…
πŸ˜„ 1
Though of course if there are any downstream steps that do an analysis across all samples then that can be an issue
I'm not very familiar with this pipeline, so I'm not sure if that's the case or not
s
I would need to use all samples for the SUPPA2 (splicing events Diff analysis in the last step of the pipeline), but maybe I can resume from there and I assume that is not so β€œheavy” as the previous steps. But very naive assumption. πŸ˜…
p
You can only resume a single run
If you do 10 smaller runs, running with all samples and
-resume
won't work, it'll just start from scratch again
s
Yep. I meant I could give the salmon results as input and run the suppa separately. The step I need to compare all conditions.
πŸ‘πŸ» 1
p
It looks like the pipeline can accept either FastQ or BAM files as input - that could be one way to reduce disk footprint
πŸ‘ 1
Trimming FastQ files takes loads of disk space, as basically all raw data is duplicated on disk
πŸ‘ 1
So you could run up to alignment in smaller batches, then submit a single run with BAM input files
assuming that I understand the pipeline logic correctly, anyway
s
I guess you do. And I will try that. Thanks a lot! =)
p
ah if you want to run suppa looks like it needs to be a transcriptome, but yeah sounds likely: https://nf-co.re/rnasplice/1.0.0/docs/usage#transcriptome-bam-files
s
Yep! Tnx!
πŸ‘πŸ» 1
p
Lycka till!
πŸŽƒ 1
j
Hello @Sarah Muniz Nardeli Sorry for just getting back to you now. I have been on annual leave for the past two weeks. What @Phil Ewels suggested will work, and is the only way I think will solve your storage issues. Thanks, James
πŸ‘ 1
s
Thanks a lot for getting back to me, James. :)