Trying to set up #CE8SSJV3N test profile to run on my org's HPC (UTokyo SHIROKANE). It's almost(!) working but fails at #C05663US2B0 with OOM error while generating plots:

ERROR ~ Error executing process > 'NFCORE_RNASEQ:RNASEQ:MULTIQC_RNASEQ:MULTIQC (multiqc_report)'

Caused by:
  Missing output file(s) `*.html` expected by process `NFCORE_RNASEQ:RNASEQ:MULTIQC_RNASEQ:MULTIQC (multiqc_report)`


Command executed:

  multiqc \
      --force \
       \
      --config 1/multiqc_config.yml --config 2/multiqc_sample_merge.yml \
      --filename multiqc_report.html \
       \
      --replace-names name_replacement.txt \
       \
      .

Command exit status:
  0

Command output:
  (empty)

Command error:

  /// MultiQC 🔍 v1.33

              config | Loading config settings from: 1/multiqc_config.yml
              config | Loading config settings from: 2/multiqc_sample_merge.yml
       version_check | MultiQC Version v1.35 now available!
         file_search | Search path: /home/ochapman/test/nf-core-rnaseq/work/28/72da4dc8c6fffcbe8a185ecde70e0d
         file_search | Only using modules: bbmap, custom_content, hisat2, kallisto, kraken, picard, preseq, qualimap, rsem, rseqc, salmon, samtools, sortmerna, star, sylphtax, umicollapse, umitools, fastqc, cutadapt, fastp, seqkit, fastqc, bowtie2, fastqc, bowtie2
              report | Skipping 5 file search patterns

               bbmap | Found 5 reports
      custom_content | biotype_counts: Found 1 samples (PlotType.BAR)
          validation | warnings while parsing lineplot.pconfig[id='dupradar'].x_lines {color: green, dash: longdash, label: {text: 0.5 RPKM}, value: 0.5, width: 1} (total messages: 2):
  • 'dash': 'LongDash' is a deprecated dash style, use 'longdash'
  • 'label': Line plot's x_lines or y_lines 'label' field is expected to be a string. Other fields other than 'text' are deprecated and will be ignored
          validation | warnings while parsing lineplot.pconfig[id='dupradar'].x_lines {color: red, dash: longdash, label: {text: 1 read/bp}, value: 1000, width: 1} (total messages: 2):
  • 'dash': 'LongDash' is a deprecated dash style, use 'longdash'
  • 'label': Line plot's x_lines or y_lines 'label' field is expected to be a string. Other fields other than 'text' are deprecated and will be ignored
      custom_content | dupradar: Found 1 samples (PlotType.LINE)
      custom_content | DupInt: Found 5 General Statistics columns
      custom_content | nf-core-rnaseq-methods-description: Found 1 sample (html)
      custom_content | salmon_deseq2_clustering: Found 5 samples (PlotType.HEATMAP)
      custom_content | salmon_deseq2_pca: Found 5 samples (PlotType.SCATTER)
      custom_content | star_salmon_deseq2_clustering: Found 5 samples (PlotType.HEATMAP)
      custom_content | star_salmon_deseq2_pca: Found 5 samples (PlotType.SCATTER)
      custom_content | strand_check_summary: Found 5 samples (PlotType.TABLE)
      custom_content | strand_check_composition: Found 2 samples (PlotType.BAR)
      custom_content | nf-core-rnaseq-summary: Found 1 sample (html)
              picard | Found 5 MarkDuplicates reports
            qualimap | Found 5 RNASeq reports
               rseqc | Found 5 read_distribution reports
               rseqc | Found 3 inner_distance reports
               rseqc | Found 5 read_duplication reports
               rseqc | Found 5 junction_annotation reports
               rseqc | Found 5 junction_saturation reports
               rseqc | Found 5 infer_experiment reports
               rseqc | Found 5 bam_stat reports
              salmon | Found 5 meta reports
              salmon | Found 5 fragment length distributions
              salmon | Found 5 library format counts reports
            samtools | Found 5 stats reports
            samtools | Found 5 flagstat reports
            samtools | Found 5 idxstats reports
                star | Found 5 reports
              fastqc | Found 8 reports
            cutadapt | Found 8 reports
              fastqc | Found 8 reports
              fastqc | Found 8 reports
         file_search | Only using modules: bbmap, picard, qualimap, rseqc, salmon, samtools, star, fastqc, cutadapt, fastqc, fastqc, biotype_counts, dupradar, nf-core-rnaseq-methods-description, sample-relationships, strandedness, nf-core-rnaseq-summary
       write_results | Rendering plots. Export plots to formats png, svg, pdf is requested, so it might take a while. To disable plot export, set `export_plots: false` in config, or remove the `--export-plots` command line flag
[OOM here]

Could be related to [this thread](https://nfcore.slack.com/archives/CE6SDBX2A/p1748889207243889?thread_ts=1748812203.507309&cid=CE6SDBX2A) but it doesn't seem to have been fixed by recent versions of MultiQC and I'm not setting

--containall

(as far as I know). Any help appreciated! System information (UTokyo SHIROKANE): Nextflow: v26.04.3 Hardware: HPC Executor: AGE Container: configured as Singularity, but that's a symlink for Apptainer on this HPC OS: Linux Version nf-core/rnaseq revision: e7ca46272c [master]

Hi, @Owen Chapman! Slack uses channels to organise discussions. Press the

...

more option on the left sidebar. Then hover over Your organisation, and finally select Channels to browse. Each nf-core pipeline has it’s own channel (e.g. #CE8SSJV3N). Please post any questions about specific pipelines in the relevant channel. If you’re not sure where to post your message, please ask @Phil Ewels for help.

Basically #CE8SSJV3N is a location in slack and not just a hashtag - better to ask there 🙂

Sure. Encountering a similar issue with multiqc in #CGB1CL0SC as well; would this be better placed in #C05663US2B0 then?

If you're using singularity, maybe try running again but with the option to disable static plot exports. That'll probably fix it.

I don't see a parameter in either rnaseq or rnafusion for a multiqc config file... would you have any guidance on (a) how to provide a multiqc config file if it's not exposed as a parameter and/or (b) an example of what that file should look like?

https://nf-co.re/rnaseq/3.26.0/parameters/#multiqc_config

--multiqc_config

It's not clear to me; have you tried allowing the multiqc process more memory?

@pontus I have, yeah. It seems to take as much memory as I care to give it; at least 64G and I haven't tried greater.

I'm not sure it is, sounds like another instance of an OOM error just from a different place in the stack

Unless you've reached your node size so that you can't really go higher, I'd give it more memory (unless you prefer doing this kind of troubleshooting, obviously).

Hmm ok I guess it's not a problem to resubmit with a giant memory request. Figured it was a different problem because (1) job logs show that resource utilization (

maxvmem

,

maxrss

) were below the job limits and the exit code is 134 (abort) and (2) disabling plotting didn't resolve the issue. Is MultiQC really expected to need more than 64G RAM on a test dataset?

Sampling logs very often don't catch usages before OOM since it's often spiky behaviour upon shifts. I'm not sure I understood the other bit, but

memory allocation of 20 bytes failed

implies memory issues quite strongly. For the dataset you get with the

test

profile I wouldn't expect 64 Gbytes usage, no. But beyond seemingly very high variation there, there can also be a lot of variation based on the system, memory pressure on the node and so on.