Hi, salmon quant is giving me some headache. I am ...
# rnaseq
p
Hi, salmon quant is giving me some headache. I am processing total RNA-seq with star_salmon on PBS, I've set the igenome GRCm38, but salmon quant always complains because it wants more RAM, like up to 384GB. I've tried to reduce the number of threads to 4, but nothing. After reads being processed it goes into std::bad_alloc, as it exhaust all memory. To reduce the input complexity I also remove ribo with sortmerna, using the default db + the pre-rRNA 45S. The sample that's failing doesn't even have a lot of ribo:
Copy code
mus_musculus_prerrna_45S_v1.fasta           0.60
    rfam-5.8s-database-id98.fasta               0.00
    rfam-5s-database-id98.fasta         0.56
    silva-arc-16s-id95.fasta            0.00
    silva-arc-23s-id98.fasta            0.00
    silva-bac-16s-id90.fasta            0.16
    silva-bac-23s-id98.fasta            0.02
    silva-euk-18s-id95.fasta            0.63
    silva-euk-28s-id98.fasta            0.31
Have you already seen this? Do you have any suggestion? Thanks! EDIT: this is the actual command:
Copy code
Command executed:

  salmon quant \
      --geneMap genes.filtered.gtf \
      --threads 4 \
      --libType=ISR \
      -t genome.transcripts.fa \
      -a IIT_363_E5_B5_S6.Aligned.toTranscriptome.out.bam \
      --reduceGCMemory \
      -o IIT_363_E5_B5_S6

  if [ -f IIT_363_E5_B5_S6/aux_info/meta_info.json ]; then
      cp IIT_363_E5_B5_S6/aux_info/meta_info.json "IIT_363_E5_B5_S6_meta_info.json"
  fi
  if [ -f IIT_363_E5_B5_S6/lib_format_counts.json ]; then
      cp IIT_363_E5_B5_S6/lib_format_counts.json "IIT_363_E5_B5_S6_lib_format_counts.json"
  fi

Command exit status:
  134

Command output:
  (empty)

Command error:
  # [ threads ] => { 4 }
  # [ libType ] => { ISR }
  # [ targets ] => { genome.transcripts.fa }
  # [ alignments ] => { IIT_363_E5_B5_S6.Aligned.toTranscriptome.out.bam }
  # [ reduceGCMemory ] => { }
  # [ output ] => { IIT_363_E5_B5_S6 }
  Logs will be written to IIT_363_E5_B5_S6/logs
  Library format { type:paired end, relative orientation:inward, strandedness:(antisense, sense) }
  [2026-08-13 01:00:02.937] [jointLog] [info] setting maxHashResizeThreads to 4
  [2026-08-13 01:00:02.937] [jointLog] [info] Fragment incompatibility prior below threshold.  Incompatible fra>
  [2026-08-13 01:00:02.955] [jointLog] [info] numQuantThreads = 2
  parseThreads = 2
  Checking that provided alignment files have consistent headers . . . done
  Populating targets from aln = "IIT_363_E5_B5_S6.Aligned.toTranscriptome.out.bam", fasta = "genome.transcripts>






  processed 0 reads in current round[2026-08-13 01:00:05.272] [jointLog] [info] replaced 3 non-ACGT nucleotides>


  processed 1000000 reads in current round

  processed 2000000 reads in current round

  processed 3000000 reads in current round[2026-08-13 01:00:26.023] [jointLog] [info]

  The alignment group queue pool has been exhausted.  3133501 extra fragments were allocated on the heap to sat>




  processed 4000000 reads in current round

  processed 5000000 reads in current round

  processed 6000000 reads in current round

  processed 7000000 reads in current round

  processed 8000000 reads in current round

  processed 9000000 reads in current round

  processed 9335963 reads in current round
  terminate called after throwing an instance of 'std::bad_alloc'
    what():  std::bad_alloc
j
Thanks for the detail. This doesn’t look like an rRNA or thread-count issue, and the queue warning itself is normally harmless.
star_salmon
currently uses Salmon 1.10.3, which has a very similar
std::bad_alloc
report on RHEL9: github.com/COMBINE-lab/salmon/issues/997 Could you share the compute node OS and whether only this sample fails? The most useful test would be running the same transcriptome BAM with a newer Salmon version. I want to update Salmon anyway, there's a fancy new Rust version that's a lot faster, I just have to find some time.
p
Thanks for the info! This is the os of a random node: Linux version 4.18.0-553.123.1.el8_10.x86_64 (mockbuild@iad1-prod-build001.bld.equ.rockylinux.org) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-28) (GCC)) #1 SMP Tue May 5 154659 UTC 2026
j
Thanks! That rules out the specific RHEL9 issue I found. It would still be worth checking the exact node used by the failed job, but this is looking more sample-specific. Did the other samples in the same run complete successfully? If so, could you compare the transcriptome BAM sizes and STAR multimapping stats? Testing the failing BAM with a newer Salmon version would also help distinguish a Salmon 1.10.3 bug from unusual alignments in this sample.
p
I think this may be the reason:
Copy code
grep -E 'Uniquely mapped reads (number|%)|% of reads unmapped' /work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_B5_S6.Log.final.out
                   Uniquely mapped reads number |       7847959
                        Uniquely mapped reads % |       6.09%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       91.07%
                     % of reads unmapped: other |       0.16%
other samples were processed fine
j
What are the stats for another sample?
p
all similar:
Copy code
/work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_B1_S4.Log.final.out
                   Uniquely mapped reads number |       7792525
                        Uniquely mapped reads % |       6.16%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       90.17%
                     % of reads unmapped: other |       0.14%
/work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_B4_S7.Log.final.out
                   Uniquely mapped reads number |       7822275
                        Uniquely mapped reads % |       5.59%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       90.64%
                     % of reads unmapped: other |       0.14%
/work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_B5_S6.Log.final.out
                   Uniquely mapped reads number |       7847959
                        Uniquely mapped reads % |       6.09%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       91.07%
                     % of reads unmapped: other |       0.16%
/work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_C1_S2.Log.final.out
                   Uniquely mapped reads number |       6082699
                        Uniquely mapped reads % |       4.74%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       92.64%
                     % of reads unmapped: other |       0.08%
/work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_C3_S3.Log.final.out
                   Uniquely mapped reads number |       3707380
                        Uniquely mapped reads % |       4.94%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       91.16%
                     % of reads unmapped: other |       0.08%
/work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_C4_S10.Log.final.out
                   Uniquely mapped reads number |       6226597
                        Uniquely mapped reads % |       5.45%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       91.09%
                     % of reads unmapped: other |       0.09%
/work/pinglese/illumina/totrna/260618_A00558_0433_BHGF7KDRX5/output_batch_1/star_salmon/log/IIT_363_E5_C6_S5.Log.final.out
                   Uniquely mapped reads number |       5149183
                        Uniquely mapped reads % |       5.42%
       % of reads unmapped: too many mismatches |       0.00%
                 % of reads unmapped: too short |       89.96%
                     % of reads unmapped: other |       0.10%
j
Those stats are pretty grim overall, but they don’t explain why this sample alone fails since the successful samples look the same. Salmon only sees the mapped transcriptome BAM anyway. Could you compare the transcriptome BAM size/record count and STAR’s
% of reads mapped to multiple loci
between B5_S6 and a successful sample? Similar unique-mapping percentages can hide a big difference in multimapping complexity, which is more likely to drive Salmon’s memory use.
p
Ok, thanks! Checking it now. I've read that maybe trimming can be the cause? github.com/alexdobin/STAR/issues/1956 I'm using
fastp
Hi Jonathan, I've been able to process all of them with the rust version of salmon. Although, there's still something wrong with these samples, as +90% are unmapped because too short. I've tried with trimgalore and they go down to 40% and without trimming stay at 90% like with fastp.
At the end the issue was that the user had communicated the wrong organism... 😭🤬
j
To summarise this in case it comes up for other folks, given the wrong-reference explanation: • STAR’s “unmapped: too short” means it couldn’t find a sufficiently long, good-scoring alignment, not that the reads themselves were necessarily short (I always forget this). Trimming changed how many reads passed that threshold, but couldn’t fix the reference mismatch. • Salmon wasn’t doing k-mer mapping here. It was reading STAR’s transcriptome BAM and estimating how the aligned reads should be assigned among transcripts. Salmon docs • The queue warning means Salmon read alignments faster than it processed them. In this run it allocated 3.1 million extra internal records before its pool filled; the code then stops further allocation, so the warning itself does not prove what caused the later crash. Salmon source •
std::bad_alloc
means Salmon 1.10.3 failed to obtain more memory, but the logs don’t establish exactly which internal structure caused it. The Rust implementation handled the same abnormal input without crashing. • SortMeRNA and the choice of trimmer were distractions. Quantification against the wrong organism would be invalid regardless of whether Salmon completed.
neon cat spin 1
p
Just a quick addition. Even with the correct organism set, only the rust version of salmon was able to finish the quantification, the current version was still giving
bad_alloc
error. The only issue remaining was that salmon failed at detecting the strandedness, due to the low quality of the library (I even removed rRNA with sortmerna, up to 40%, but nothing).
👍 1