I'm running the rnaseq pipeline on my school's HPC using Apptainer, but I keep running into problems specifically with NFCORE_RNASEQRNASEQQUANTIFY_BAM_SALMON:SALMON_QUANT. It looks like every other task completed, and about half of the samples have gotten through this task, but some keep failing. I've allocated plenty of memory to this task and I'm not out of storage space, but I think the HPC has a virtual memory limit of 50GB. Would the virtual memory limit explain these failures for some samples? If not, does anyone have any suggestions? Thanks!

error [nextflow.exception.ProcessFailedException]: Process `NFCORE_RNASEQ:RNASEQ:QUANTIFY_BAM_SALMON:SALMON_QUANT (s0095138)` terminated with an error exit status (1)
May-05 20:58:39.275 [TaskFinalizer-3] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_RNASEQ:RNASEQ:QUANTIFY_BAM_SALMON:SALMON_QUANT (s0095138)'

Caused by:
  Process `NFCORE_RNASEQ:RNASEQ:QUANTIFY_BAM_SALMON:SALMON_QUANT (s0095138)` terminated with an error exit status (1)


Command executed:

  salmon quant \
      --geneMap gencode.v49.primary_assembly.annotation.filtered.gtf \
      --threads 8 \
      --libType=ISR \
      -t genome.transcripts.fa \
      -a s0095138.Aligned.toTranscriptome.out.bam \
       \
      -o s0095138

  if [ -f s0095138/aux_info/meta_info.json ]; then
      cp s0095138/aux_info/meta_info.json "s0095138_meta_info.json"
  fi
  if [ -f s0095138/lib_format_counts.json ]; then
      cp s0095138/lib_format_counts.json "s0095138_lib_format_counts.json"
  fi

Command exit status:
  1

Command output:
  (empty)

Command error:
  Total # of mapped reads : 40802512
  # of uniquely mapped reads : 3552413
  # ambiguously mapped reads : 37250099



  [2026-05-05 20:57:06.666] [jointLog] [info] Computed 480904 rich equivalence classes for further processing
  [2026-05-05 20:57:06.666] [jointLog] [info] Counted 39850892 total reads in the equivalence classes
  [2026-05-05 20:57:06.666] [jointLog] [info] starting optimizer
  [2026-05-05 20:57:06.905] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
  [2026-05-05 20:57:06.931] [jointLog] [info] iteration = 0 | max rel diff. = 1583.19
  [2026-05-05 20:57:09.665] [jointLog] [info] iteration = 100 | max rel diff. = 20.5892
  [2026-05-05 20:57:12.529] [jointLog] [info] iteration = 200 | max rel diff. = 14.0013
  [2026-05-05 20:57:14.973] [jointLog] [info] iteration = 300 | max rel diff. = 2.03673
  [2026-05-05 20:57:17.529] [jointLog] [info] iteration = 400 | max rel diff. = 3.81979
  [2026-05-05 20:57:20.607] [jointLog] [info] iteration = 500 | max rel diff. = 4.45175
  [2026-05-05 20:57:23.302] [jointLog] [info] iteration = 600 | max rel diff. = 12.7871
  [2026-05-05 20:57:26.003] [jointLog] [info] iteration = 700 | max rel diff. = 5.81501
  [2026-05-05 20:57:28.488] [jointLog] [info] iteration = 800 | max rel diff. = 2.47634
  [2026-05-05 20:57:30.983] [jointLog] [info] iteration = 900 | max rel diff. = 0.661868
  [2026-05-05 20:57:33.767] [jointLog] [info] iteration = 1000 | max rel diff. = 0.942412
  [2026-05-05 20:57:36.588] [jointLog] [info] iteration = 1100 | max rel diff. = 3.70971
  [2026-05-05 20:57:39.387] [jointLog] [info] iteration = 1200 | max rel diff. = 0.313561
  [2026-05-05 20:57:42.373] [jointLog] [info] iteration = 1300 | max rel diff. = 0.392029
  [2026-05-05 20:57:45.425] [jointLog] [info] iteration = 1400 | max rel diff. = 0.105514
  [2026-05-05 20:57:48.175] [jointLog] [info] iteration = 1500 | max rel diff. = 0.564924
  [2026-05-05 20:57:51.014] [jointLog] [info] iteration = 1600 | max rel diff. = 0.119491
  [2026-05-05 20:57:53.667] [jointLog] [info] iteration = 1700 | max rel diff. = 0.996128
  [2026-05-05 20:57:56.490] [jointLog] [info] iteration = 1800 | max rel diff. = 0.181462
  [2026-05-05 20:57:59.342] [jointLog] [info] iteration = 1900 | max rel diff. = 0.726279
  [2026-05-05 20:58:02.364] [jointLog] [info] iteration = 2000 | max rel diff. = 0.135374
  [2026-05-05 20:58:05.177] [jointLog] [info] iteration = 2100 | max rel diff. = 0.078895
  [2026-05-05 20:58:07.815] [jointLog] [info] iteration = 2200 | max rel diff. = 0.0613483
  [2026-05-05 20:58:10.737] [jointLog] [info] iteration = 2300 | max rel diff. = 0.0387256
  [2026-05-05 20:58:13.630] [jointLog] [info] iteration = 2400 | max rel diff. = 0.667499
  [2026-05-05 20:58:16.141] [jointLog] [info] iteration = 2500 | max rel diff. = 0.513678
  [2026-05-05 20:58:18.950] [jointLog] [info] iteration = 2600 | max rel diff. = 0.285459
  [2026-05-05 20:58:21.479] [jointLog] [info] iteration = 2700 | max rel diff. = 7.03545
  [2026-05-05 20:58:22.713] [jointLog] [info] iteration = 2747 | max rel diff. = 0.0063784
  [2026-05-05 20:58:22.722] [jointLog] [info] finished optimizer
  [2026-05-05 20:58:22.722] [jointLog] [info] writing output

  Freeing memory used by read queue . . .
  Joined parsing thread . . . "s0095138.Aligned.toTranscriptome.out.bam"
  Closed all files . . .
  Emptied frag queue. . .
  Emptied Alignment Group Pool. .
  Emptied Alignment Group Queue. . . done
  [2026-05-05 20:58:27.866] [jointLog] [info] Computing gene-level abundance estimates
  Error allocating memory.

50 GB should be more than enough.

SALMON_QUANT

has the

label "process_medium"

, which allocates 36 GB by default. If the process is being retried (which it does on certain exit codes, but not on

1

as in this case), it would double that -- up to the setting of

memory

in

resourceLimits

(which you should make sure is set to

50.GB

). I don't know if your problem is really the 36 GB limit or not, I would look at entry for this process in the table in the

execution_report_*.html

file and see if the

peak_rss

is getting close to the limit (

allocated memory

). If you want to try with more, you can either set the

errorStrategy

to

retry

(either conditionally for the exit code as it is by default, you would just have to add

1

to the list, or generally, i.e. independent of the reason for failure), or you could allocate

50.GB

to the process in first place (using

withName

). You would either of those in a config file to be used with

-c

. 🤞