Hi, I hope it's okay to ask my stupid questions here. I tried to run the nascent pipeline on Dardel recently, but it could not be executed successfully. I used the following code: export NXF_SINGULARITY_CACHEDIR=/path/to/my/dir/ module load PDC java singularity nextflow run nf-core/nascent -profile pdc_kth --project=naiss2025-22-154 --input /cfs/klemming/projects/supr/naiss2025-22-154/Proseqfastq/combine/Proseq1.csv --outdir /cfs/klemming/projects/supr/naiss2025-22-154/Proseqfastq/result1/ --email tarazhang@foxmail.com --assay_type PROseq --genome GRCm38 -r 2.3.0 -work-dir /cfs/klemming/projects/supr/naiss2025-22-154/nfcore/ --save_reference FALSE --scratch=TRUE The pipeline will always encounter some error and shows the information below ERROR ~ Error executing process > 'NFCORE_NASCENTNASCENTCOVERAGE_GRAPHS:BEDTOOLS_GENOMECOV_MINUS (SA_5m_REP3)' Caused by: Process

NFCORE_NASCENT:NASCENT:COVERAGE_GRAPHS:BEDTOOLS_GENOMECOV_MINUS (SA_5m_REP3)

terminated for an unknown reason -- Likely it has been terminated by the external system Command executed: bedtools \ genomecov \ -ibam SA_5m_REP3.sorted.bam \ -bg -strand - \ | bedtools sort \ > SA_5m_REP3.minus.bedGraph cat <<-END_VERSIONS > versions.yml "NFCORE_NASCENTNASCENTCOVERAGE_GRAPHS:BEDTOOLS_GENOMECOV_MINUS": bedtools: $(bedtools --version | sed -e "s/bedtools v//g") END_VERSIONS Command exit status: - Command output: (empty) Command error: INFO: Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred INFO: Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred INFO: Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred INFO: fuse2fs not found, will not be able to mount EXT3 filesystems INFO: gocryptfs not found, will not be able to use gocryptfs Work dir: /cfs/klemming/projects/supr/naiss2025-22-154/nfcore/56/e86e77f3009640f45b17b64a3f87f1 Container: /cfs/klemming/projects/supr/naiss2025-22-154/Proseqfastq/cache/depot.galaxyproject.org-singularity-bedtools-2.31.1--hf5e1c6e_0.img Tip: you can replicate the issue by changing to the process work dir and entering the command

bash .command.run

-- Check '.nextflow.log' file for details ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting -- Check '.nextflow.log' file for details ERROR ~ Failed to invoke

workflow.onComplete

event handler -- Check script '.nextflow/assets/nf-core/nascent/subworkflows/local/utils_nfcore_nascent_pipeline/main.nf' at line: 130 or see '.nextflow.log' file for more details I checked the log, and it didn't give me any useful information. It would be much appreciated if someone could help me solve this issue.

can you check that you are not out of space - run

projinfo

?

Hi, when the pipeline crashed, the project folder still had around 300 GB space left

My suspicion is that SLURM cancelled your job and if you do not see traces of OOM then probably is short of time - you might need to increase the time manually. If you find the jobid you can check with

seff jobID

for some info from SLURM. Otherwise - best is to contact PDC to assist you since they can get closer to the problem...

Thank you!! I will contact them. Meanwhile, I wonder if it is possible to apply for more time while I try to run the pipeline? Cause when the pipeline started, it showed invalid inputs as below. WARN: The following invalid input values have been detected: * --project: naiss2025-22-154 * --scratch: true * --max_memory: 1.7 TB * --max_cpus: 256 * --max_time: 7d * --schema_ignore_params: genomes,input_paths,cluster-options,clusterOptions,project,validationSchemaIgnoreParams * --validationSchemaIgnoreParams: genomes,input_paths,cluster-options,clusterOptions,project,schema_ignore_params

I often run this if I don't get an exit code:

sacct --user=$USER -S now-7days --format=JobID,JobName%50,State,ExitCode,Start,End  | grep -w -v "batch\|extern"

It shows out of memory, is there anything I can do to increase the max memory?

You increase memory / time / cpus etc for a specific process by doing:

process {
    withName: 'PROCESS_NAME' {
        memory = 500.GB
        time   = 2.d
    }
}

in a

nextflow.config

where you're running the pipeline or calling it

custom.config

and then doing

nextflow run ... -c custom.config

Thanks for the reply, I've contacted with pdc support, turns out the cause might be "My wild guess at this moment is that the problematic tasks actually specifies some low memory value in their own process definitions" I wonder if it's possible to set a minimal memory to test and avoid the problem?

If you're using the

pdc_kth

profile, it does set a minimum memory, otherwise the resources scale with cpus/memory allocated by the default labels

Yes， I'm using the pdc_kth profile according to the nfcore pdc page, but it seems not to be working, or maybe I didn't set the profile correctly

You don't need to copy that file anywhere if you're using an nf-core pipeline. You add

-profile pdc_kth

to your

nextflow run ...

command

nextflow run nf-core/nascent -profile pdc_kth --project=naiss2025-22-154 --input "/cfs/klemming/projects/supr/naiss2025-22-154/Proseqfastq/combine/Proseq1.csv" --outdir /cfs/klemming/projects/supr/naiss2025-22-154/Proseqfastq/result1/ --email tarazhang@foxmail.com --assay_type PROseq --genome GRCm38 -r 2.3.0 -work-dir /cfs/klemming/projects/supr/naiss2025-22-154/nfcore/ --save_reference FALSE --scratch=TRUE -c /cfs/klemming/home/d/dhrubatm/.nextflow/custom.config --aligner star That's what I did....

I don't follow, does your own config file that you specify there increase the amount of memory for the process? (It can generally be that the amount of memory needed for a process varies wildly with the type of data.)

process { withName: 'NASCENT' { memory = 1524.GB time = 2.d } } At first, I thought it might be caused by insufficient memory, so I set up a custom.config file to ensure the task has enough memory. But the pipeline still crashed, and I checked those job labeled with "Out of memory", usually they do not require a lot of memory to run. After talking to the PDC support, they guess the problem might be caused by "the tasks actually specifies some low memory value in their own process definitions" That's why I would like to try to set up a minimal memory for the pipeline, even though it is presented in the kth_pdc profile.

I don't recall the rules now, are you certain that matches?

That won't match anything. The workflow is called

NASCENT

but

withName:

selector must match a process name, e.g.

process {
    withName: 'BEDTOOLS_GENOMECOV_MINUS' {
        memory = 500.GB
        time   = 2.d
    }
}

nf-core has also hidden those selector warnings so the user doesn't see if they supply the wrong name to a

withName:

selector.

Thank you guys! The problem is solved after I updated the config according to Mahesh suggestion.