Hello, I am running the scnanoseq pipeline on an HPC system and I am encountering some issues with Java heap space during FASTQC_NANOPLOT_PRE_TRIM:FASTQC step. The pipeline consistently fails with a

java.lang.OutOfMemoryError

.

ERROR ~ Error executing process > 'NFCORE_SCNANOSEQ:SCNANOSEQ:FASTQC_NANOPLOT_PRE_TRIM:FASTQC (BEB19013)'
 
Caused by:
 Process `NFCORE_SCNANOSEQ:SCNANOSEQ:FASTQC_NANOPLOT_PRE_TRIM:FASTQC (BEB19013)` terminated with an error exit status (3)
 
 
Command executed:
 
 printf "%s %s\n" BEB19013.merged.fastq.gz BEB19013.raw.gz | while read old_name new_name; do
   [ -f "${new_name}" ] || ln -s $old_name $new_name
 done
  
 fastqc \
    \
   --threads 6 \
   BEB19013.raw.gz
  
 cat <<-END_VERSIONS > versions.yml
 "NFCORE_SCNANOSEQ:SCNANOSEQ:FASTQC_NANOPLOT_PRE_TRIM:FASTQC":
   fastqc: $( fastqc --version | sed '/FastQC v/!d; s/.*v//' )
 END_VERSIONS
 
Command exit status:
 3
 
Command output:
 application/gzip
 Terminating due to java.lang.OutOfMemoryError: Java heap space
 
Command error:
 Started analysis of BEB19013.raw.gz
 Approx 5% complete for BEB19013.raw.gz
 Approx 10% complete for BEB19013.raw.gz
 Approx 15% complete for BEB19013.raw.gz
 Approx 20% complete for BEB19013.raw.gz
 Approx 25% complete for BEB19013.raw.gz
 Approx 30% complete for BEB19013.raw.gz
 Approx 35% complete for BEB19013.raw.gz
 Approx 40% complete for BEB19013.raw.gz
 Approx 45% complete for BEB19013.raw.gz
 
Work dir:
 /data/project/rizzardilab/drbhatta/long_read/scnanoseq/D13_SUP_PDD/work/7b/b6d20f015d9b46b2c8e30026d5601a
 
Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
 
 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>
 
 -- Check '.nextflow.log' file for details

Command Used:

export NXF_OPTS='-Xms1g -Xmx4g
nextflow run nf-core/scnanoseq -r dev \
 --params-file params.yaml \
--split_amount: 200000 \
 -profile cheaha

While the pipeline is running, I can see two jobs submitted with the following allocations: • 36 GB of memory and 6 CPUs for one job. • 12 GB of memory and 2 CPUs for another. I have not tried using a custom config file yet. I am running this pipeline on just one sample from a PromethION flow cell, which contains 177M reads. Any insights or suggestions would be greatly appreciated! Thank you in advance for your help.

Hi @Dev Raj Bhattarai, Thank you for using scnanoseq! So I would definitely recommend a custom config in this case. We've noted that FastQC definitely is requiring much more memory with some of the long-read sequenced data, especially when it involves PromethION. We have an example custom config that may be a good starting point for your analysis in our README towards the bottom of the Troubleshooting section here: https://github.com/nf-core/scnanoseq?tab=readme-ov-file#troubleshooting It includes the process that runs FastQC, as well as a few other processes that we have noted having issues with these larger datasets

@Dev Raj Bhattarai: agree with Austyn and as and as a side note since you are using the cheaha profile, I would recommend at this time to use a modified version of that. See below: Austyn and I have noticed that there has been a reduction in the number of nodes available in the

largemem-long

partition. As such, your job may end up being in the queue for much longer than expected. (we may eventually push a change to the profile to remove that partition or change the logic a bit. But we are still evaluating that) I would instead (at this time) suggest to use the following modified version of the profile which has been added to the custom config (see attached). This already includes some of the process where we typically change resource limits, but note that you may need to increase them further if needed (these limits have worked with promethion data for up to 123 M as a reference). Also be sure you are using the version of the pipeline which contains IsoQuant > 3.5.0. [with this you would remove

-profile cheaha

and add

-c custom.conf

instead to your running command]. Let us know if you need more clarifications or help.

Thank you so much for the response. I will try this and let you guys know.