https://nf-co.re logo
Join Slack
Powered by
Hi all, the pipeline failed with an out of memory ...
# riboseq
r
Hi all, the pipeline failed with an out of memory (137) exit error
Copy code
[21/7d3f31] NOTE: Process `NFCORE_RIBOSEQ:RIBOSEQ:FASTQ_QC_TRIM_FILTER_SETSTRANDEDNESS:SORTMERNA (26099TOTAL_RNA)` terminated with an error exit status (137) -- Execution is retried (2)
could you please advice on how I can increase the memory for this and other memory-hungry steps? thank you so much for your kind help
j
SORTMERNA
is labelled 
process+high
which according to conf/base.config gives it 72GB memory. I may be wrong but my intuition would say it should not need more memory than that. You can of course change that memory param locally and see if it helps but I suspect you may need to look closer at the logs.
r
ok that's good intel, thanks Jack
j
I did see another GitHub issue after this that cited OOM at 240GB so my intuition could well be wrong
Or that issue was also misdiagnosing the issue 🤷‍♂️
r
hey Jack, I tried again increasing the memory
Copy code
process {
  withName: SORTMERNA {
    memory = '100 GB'
  }
}
but it's still failing with the same error
is there anyway of bypassing this or making it more memory-efficient?
I did see that github issue as well, same error as me
for now I'll try it with 
--remove_ribo_rna False
and see if I can run the pipeline successfully. In your experience, is the SORTMERNA step crucial? I would assume STAR can just remove rRNA reads as multi-mappers no?
j
I dont think the STAR params are strict enough to rely on that. rRNA removal is quite important if you are going to do count based analysis like differential translation stuff. Could you try something for me? I think changing 
--num_alignments 1
to 
--no-best
might solve the issue and since we dont care about the actual rRNA alignment we dont lose any valuable info. Can you make the change on this line and report back? https://github.com/nf-core/riboseq/blob/235e008eda637ff7827db31669148824e9dd1ede/conf/modules.config#L360
r
Hi Jack, sorry for the late reply, I was on Easter holidays. I'll try and let you know!
Hey Jack, I changed the line that you suggested, and now sortmerna seems to run successfully:
Copy code
cat nf-log_3482489.out | grep FTER_SORTMERNA | grep "12 of"
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
[71/1330a3] NFC…FTER_SORTMERNA (540_99RPF) | 12 of 12 ✔
nf-log_3482489.out
the pipeline now fails at ribotricer
Open in Slack
PreviousNext
Powered by