Hello, I am attempting to run the #C038XMTQWBT workflow but keep running into a process error with hifiasm. I have pacbio hifi reads on hand, no reference, no short reads nor annotation files. I am running in an interactive session on a pbs cluster with a node that has 16 CPU cores. It seems hifiasm requires 24 CPUs by default and as such I get a

Process requirement exceeds available CPUs -- req: 24; avail: 16

error. Is there a way to pass 16 cores globally to all processes in the workflow? Command executed

nextflow run nf-core/genomeassembler -r dev -name PS-FT -profile apptainer -work-dir [REDACTED] -resume -params-file params.json

Where

params.json

is;

{
  "input": "samplesheet.csv",
  "outdir": "PS-FT-RESULT",
  "hifi": true,
  "assembler": "hifiasm",
  "genome_size": 2300000000,
  "lift_annotations": false,
  "use_ref": false,
  "flye_mode": ""
}

samplesheet.csv

sample,ontreads,hifireads,ref_fasta,ref_gff,shortread_F,shortread_R,paired
sample,,[REDACTED_PATH].fastq.gz,,,,,false

Any advice is greatly appreciated. Thank you

Note you wouldn't want to pass the same amount to all processes in teh workflow

You likely need to configure the pipeline to your available computational resources to you machine or infrastructure. Please see the following two(!) sections: - https://nf-co.re/docs/running/configuration/nextflow-for-your-system#workflow-resources - https://nf-co.re/docs/running/configuration/nextflow-for-your-system#set-max-resources of the documentation for more information.

Tha'ts better :_)

Hi James, thank you for the feedback. I'm very new to using nextflow and nf-core workflows. I edited the base.config file to 16 CPUs for processes with label high and now the pipeline is running.

That works until a process fails and is retried with increased CPU allocation. Instead, you should provide a custom config file with

-c

that contains your system-specific resource limits:

process {
    resourceLimits = [
        cpus: XX,
        memory: 'XX.GB',
        time: 'XX.h'
    ]
}

Thank you Alex, that's a great suggestion. I tried the suggested syntax and got the following error upon execution

ERROR ~ Unable to parse config file: '[REDACTED]/limits.config'

  Compile failed for sources FixedSetSources[name='/groovy/script/Script987A2597DCF9FE4B26D1A71528D4BC29/_nf_config_fa8064bf']. Cause: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
  /groovy/script/Script987A2597DCF9FE4B26D1A71528D4BC29/_nf_config_fa8064bf: 1: Unexpected input: '{' @ line 1, column 9.
process {
        ^   
  
  1 error


 -- Check '.nextflow.log' file for details

Can you please post the content of your

limits.config

?

Hi Alex, the contents are as follows;

process {
    resourceLimits = [
        cpus: 16,
        memory: 110.GB,
        time: 23.h
    ]
}

You might need quotes around the memory and time entries

Thank you! I'll try that out later today and keep you all posted.

I'm not 💯 sure tho

No, quotes are not needed. Could it be a Windows line ending problem?