Anyone familiar with getting MASHWrapper running on nextflow core pipelines, working with paired illumina reads and struggling with toggling between parallelized MASH results and a separate process for sequence pairs to run via mashwrapper. https://github.com/tiagofilipe12/mash_wrapper

Quick question about expected scheduling behaviour. I’m seeing downstream processes wait until all upstream tasks for a sample set finish before any downstream tasks start. For example:

executor >  slurm (6)
[4d/0abfaf] YPE…RBASE:FASTQC (sub_tes33d6) | 0 of 3
[6a/6d8f03] YPE…BASE:FASTP (sub_spike0125) | 2 of 3
[-        ] YPE…NCLE_FGBIO_PERBASE:BWA_MEM -

executor >  slurm (6)
[2a/cfa7f3] YPE…ASE:FASTQC (sub_spike0125) | 1 of 3
[6a/6d8f03] YPE…BASE:FASTP (sub_spike0125) | 2 of 3
[-        ] YPE…NCLE_FGBIO_PERBASE:BWA_MEM -

executor >  slurm (9)
[4d/0abfaf] YPE…RBASE:FASTQC (sub_tes33d6) | 3 of 3 ✔
[fd/92aa1d] YPE…ERBASE:FASTP (sub_tes33d6) | 3 of 3 ✔
[50/b7a700] YPE…BASE:BWA_MEM (sub_tes33d6) | 0 of 3

It looks like BWA_MEM only starts once all 3 FASTP tasks for that sample have finished, rather than as each FASTP task completes. Is that expected Nextflow behaviour? Or is there a way to configure Nextflow (or Slurm/executor) to let downstream tasks be scheduled as each upstream task finishes (like a per-task streaming)?

This message was deleted.

Hi everyone, I was processing around 500 samples of 16S data using the nf-core/ampliseq pipeline on our university HPC. The run took more than two days and eventually stopped due to a connection timeout. When I tried to restart the run, I got the following error:

(nextflow) icastro@sbcb-cbiot-05:~$ nextflow run nf-core/ampliseq -profile conda,test --outdir test_ampliseq

N E X T F L O W   ~  version 25.04.3
nextflow run nf-core/ampliseq -profile conda,test --outdir test_ampliseq
ERROR ~ Unable to parse config file: '/home/icastro/.nextflow/assets/nf-core/ampliseq/nextflow.config'

  Cannot read config file include: <https://raw.githubusercontent.com/nf-core/configs/master/pipeline/ampliseq.config>

The same error now happens with any nf-core pipeline I try to run, for example:

(nextflow) icastro@sbcb-cbiot-05:~$ nextflow run nf-core/sarek -profile conda,test --outdir test

 N E X T F L O W   ~  version 25.04.3

ERROR ~ Unable to parse config file: '/home/icastro/.nextflow/assets/nf-core/sarek/nextflow.config'

  Cannot read config file include: <https://raw.githubusercontent.com/nf-core/configs/master/nfcore_custom.config>

It seems Nextflow can’t fetch the nf-core configs from GitHub anymore. Has anyone experienced this issue or knows how to fix it? I’ve already tried removing and reinstalling nf-core pipelines with

nextflow pull

, but the error persists. Thanks in advance for your help! 🙏🏼

Hello, I have small problem in long-read pipeline. I would love to get advices. This is my starting code:

nextflow run nf-core/mag \

-profile apptainer \

--input samplesheet.csv \

--outdir ./mag_output \

--binqc_tool checkm2 \

--gtdb_db /db/GTDB/r226/gtdbtk_r226_data.tar.gz \

-r 5.0.0 \

-resume

The pipeline works great until PROKKA. Then this error pops up. I could not solve it, is there any advice for me? Error msg: -[nf-core/mag] Pipeline completed with errors- ERROR ~ Error executing process > 'NFCORE_MAG🔍PROKKA (METAMDBG-CONCOCT-PAEA-B4_9)' Caused by: Process

NFCORE_MAG:mag:PROKKA (METAMDBG-CONCOCT-PAEA-B4_9)

terminated with an error exit status (2) Command executed: prokka \ --metagenome \ --cpus 2 \ --prefix METAMDBG-CONCOCT-PAEA-B4_9 \ \ \ METAMDBG-CONCOCT-PAEA-B4_9.fa cat <<-END_VERSIONS > versions.yml "NFCORE_MAG🔍PROKKA": prokka: $(echo $(prokka --version 2>&1) | sed 's/^.*prokka //') END_VERSIONS Command exit status: 2 Command output: (empty) Command error: [145315] Determined blastp version is 002016 from 'blastp: 2.16.0+' [145315] Looking for 'cmpress' - found /opt/conda/bin/cmpress [145315] Determined cmpress version is 001001 from '# INFERNAL 1.1.5 (Sep 2023)' [145315] Looking for 'cmscan' - found /opt/conda/bin/cmscan [145315] Determined cmscan version is 001001 from '# INFERNAL 1.1.5 (Sep 2023)' [145315] Looking for 'egrep' - found /usr/bin/egrep [145315] Looking for 'find' - found /usr/bin/find [145315] Looking for 'grep' - found /usr/bin/grep [145315] Looking for 'hmmpress' - found /opt/conda/bin/hmmpress [145315] Determined hmmpress version is 003004 from '# HMMER 3.4 (Aug 2023); http://hmmer.org/' [145315] Looking for 'hmmscan' - found /opt/conda/bin/hmmscan [145315] Determined hmmscan version is 003004 from '# HMMER 3.4 (Aug 2023); http://hmmer.org/' [145315] Looking for 'java' - found /opt/conda/bin/java [145315] Looking for 'makeblastdb' - found /opt/conda/bin/makeblastdb [145315] Determined makeblastdb version is 002016 from 'makeblastdb: 2.16.0+' [145315] Looking for 'minced' - found /opt/conda/bin/minced [145315] Determined minced version is 004002 from 'minced 0.4.2' [145315] Looking for 'parallel' - found /opt/conda/bin/parallel [145316] Determined parallel version is 20241122 from 'GNU parallel 20241122' [145316] Looking for 'prodigal' - found /opt/conda/bin/prodigal [145316] Determined prodigal version is 002006 from 'Prodigal V2.6.3: February, 2016' [145316] Looking for 'prokka-genbank_to_fasta_db' - found /opt/conda/bin/prokka-genbank_to_fasta_db [145316] Looking for 'sed' - found /usr/bin/sed [145316] Looking for 'tbl2asn' - found /opt/conda/bin/tbl2asn [145316] Determined tbl2asn version is 025007 from 'tbl2asn 25.7 arguments:' [145316] Using genetic code table 11. [145316] Loading and checking input file: METAMDBG-CONCOCT-PAEA-B4_9.fa [145316] Wrote 1 contigs totalling 5578 bp. [145316] Predicting tRNAs and tmRNAs [145316] Running: aragorn -l -gc11 -w METAMDBG\-CONCOCT\-PAEA\-B4_9\/METAMDBG\-CONCOCT\-PAEA\-B4_9\.fna [145316] Found 0 tRNAs [145316] Predicting Ribosomal RNAs [145316] Running Barrnap with 2 threads [145316] Found 0 rRNAs [145316] Skipping ncRNA search, enable with --rfam if desired. [145316] Total of 0 tRNA + rRNA features [145316] Searching for CRISPR repeats [145316] Found 0 CRISPRs [145316] Predicting coding sequences [145316] Contigs total 5578 bp, so using meta mode [145316] Running: prodigal -i METAMDBG\-CONCOCT\-PAEA\-B4_9\/METAMDBG\-CONCOCT\-PAEA\-B4_9\.fna -c -m -g 11 -p meta -f sco -q [145316] Found 5 CDS [145316] Connecting features back to sequences [145316] Not using genus-specific database. Try --usegenus to enable it. [145316] Annotating CDS, please be patient. [145316] Will use 2 CPUs for similarity searching. [145317] There are still 5 unannotated CDS left (started with 5) [145317] Will use blast to search against /opt/conda/db/kingdom/Bacteria/IS with 2 CPUs [145317] Running: cat METAMDBG\-CONCOCT\-PAEA\-B4_9\/METAMDBG\-CONCOCT\-PAEA\-B4_9\.IS\.tmp\.67\.faa | parallel --gnu --plain -j 2 --block 366 --recstart '>' --pipe blastp -query - -db /opt/conda/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > METAMDBG\-CONCOCT\-PAEA\-B4_9\/METAMDBG\-CONCOCT\-PAEA\-B4_9\.IS\.tmp\.67\.blast 2> /dev/null [145317] Could not run command: cat METAMDBG\-CONCOCT\-PAEA\-B4_9\/METAMDBG\-CONCOCT\-PAEA\-B4_9\.IS\.tmp\.67\.faa | parallel --gnu --plain -j 2 --block 366 --recstart '>' --pipe blastp -query - -db /opt/conda/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > METAMDBG\-CONCOCT\-PAEA\-B4_9\/METAMDBG\-CONCOCT\-PAEA\-B4_9\.IS\.tmp\.67\.blast 2> /dev/null Work dir: /scratch/shire/data/nj/projects/mosquito_ubiome_aging/preliminary/20251022_aytac_trial_analysis/work/64/757b4928116431c1597396b80a1475 Container: /gen/lnxdata/nf-core/community.wave.seqera.io-library-prokka_openjdk-10546cadeef11472.img Tip: when you have fixed the problem you can continue the execution adding the option

-resume

to the run command line -- Check '.nextflow.log' file for details ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting -- Check '.nextflow.log' file for details [Mon 27 Oct 184010 CET 2025] Finished workflow.

Hi everyone! Could someone please clarify whether the output file

salmon.merged.gene_counts.tsv

is already normalized in any way? Thank you!

Please somebody helps me to figure out why I get this error when running rnaseq pipeline on one sample

Caused by:

`Process

NFCORE_RNASEQ:RNASEQ:FASTQ_QC_TRIM_FILTER_SETSTRANDEDNESS:FQ_LINT (S1)

terminated with an error exit status (1)`

Command executed:

fq lint \

--disable-validator P001 \

034_1_S1_R1_001.fastq.gz 034_1_S1_R2_001.fastq.gz > S1.fq_lint.txt

cat <<-END_VERSIONS > versions.yml

"NFCORE_RNASEQ:RNASEQ:FASTQ_QC_TRIM_FILTER_SETSTRANDEDNESS:FQ_LINT":

fq: $(echo $(fq lint --version | sed 's/fq-lint //g'))

END_VERSIONS

Command exit status:

1

this this my code

nextflow run nf-core/rnaseq \

-profile singularity \

-process.executor slurm \

--input /users/fi0001/singlesample.csv \

--outdir /parallel_scratch/$USER/nfcore/rnaseq_S1/results \

--genome GRCh38 \

--start_from_fastq_qc false \

--skip_fq_lint true \

--save_trimmed \

-work-dir /parallel_scratch/$USER/nfcore/rnaseq_S1/work \

-with-report -with-trace -with-timeline

Hi... has anyone used nf-core/seqcoverage ? do you know if this still working.. I am running into issues like "WARN: Cannot read project manifest -- Cause: Remote resource not found: https://api.github.com/repos/nf-core/seqcoverage/contents/nextflow.config Remote resource not found: https://api.github.com/repos/nf-core/seqcoverage/contents/main.nf Any help will be appreciated.

Hi, In my pipeline I suddenly got a strange result where it appears that two samples got mixed up halfway through the pipeline. It seems that the wrong meta.id was attached to the wrong file from SAMTOOLS_DEPTH. Even though I don't think that's possible. I did a number of "-resumes" during the run, but I don't have the work-directory anymore so I can't exactly trace down the error. But my questions is, are there tools (e.g. in nf-core tools) or methods to check that a pipeline does not have any wrong logic or other errors that can allow these things to happen? For example, trace every single file and how they pass through the pipeline?

Hi everyone, I’m new user to Nextflow. I’m running the dev version of

nf-core/genomeassembler

with Apptainer on our SLURM cluster (Leftraru) in Chile, but I keep running into a consistent error during the container pulling phase. Error message:

Cannot invoke "nextflow.util.Duration.toMillis()" because "this.pullTimeout" is null

I’ve tried setting

apptainer.pullTimeout

at the user level (e.g., in

.config

and through environment variables), but the error persists. 1. Has anyone seen this specific

pullTimeout is null

error before? Was it fixed by having the cluster admin define a default value in the global Nextflow config? 2. Is there a known user-level workaround to force initialization of this variable when the system default is missing? I’ve already contacted our cluster support team, but I’m hoping to get some quick insights from the community in the meantime. Thanks a lot for any guidance — still learning my way around Nextflow!

Hi Team, I'm trying to run mashwrapper nextflow pipeline from the CDC inside an existing nf-core pipeline. Any advice for running outside nextflow pipelines or modules in your own nextflow code? Best, Dylan

Hello, I hope you are doing well. I would like to ask for your feedback on the approach I used to compare transcriptional responses between strains. For each stress condition, I identified differentially expressed genes and examined the pairwise overlap of up-regulated and down-regulated genes between two strains at a time, while excluding genes that were also regulated in the third strain under the same condition. To determine whether the observed overlaps reflected coordinated regulation rather than coincidence, I calculated the odds ratio and the corresponding Fisher’s exact test p-value for each pairwise comparison. This analysis was focused only on the number of shared versus strain-specific DEGs, without considering functional pathways or enrichment analysis at this stage. I would like to confirm whether this approach is appropriate for assessing similarity and divergence in transcriptional responses based purely on DEG overlap counts? Thank you

Hi all! I haven't foud yet similar topics, so in the meantime I'll drop a request for help here: I'm trying to update modules in a pipeline, I had no issues with the first two but then I'm only gettinr an error that I think is related to a python version. I can't paste the whole output but I have it, if needed I have latest versione of nextflow and nf-core/tools. And I have Python 3.13 ... please send help

(new-dev) nadiunix@LAPTOP-FAG8G0FQ:~/sammyseq$ nf-core modules update untar
...
TypeError: unhashable type: 'dict'

Question.. my pipeline stopped midway because I ran out of storage. Is there a way to resume from where the pipeline stopped? executor > local (393) [- ] NFC…DEX_BISMARK_BWAMETH:GUNZIP - [7c/6feebd] NFC…ex/reference_genome.fasta) | 1 of 1 ✔️ [f9/d43039] NFC…AT_FASTQ (SYNSC-741point2) | 57 of 57 ✔️ [81/777e70] NFC…Q:FASTQC (SYNSC-738point2) | 57 of 57 ✔️ [7a/0f6c02] NFC…IMGALORE (SYNSC-741point2) | 57 of 57 ✔️ [aa/bdf0cd] NFC…RK_ALIGN (SYNSC-741point2) | 57 of 57 ✔️ [0c/de53d8] NFC…UPLICATE (SYNSC-737point4) | 56 of 57 [f2/23fc15] NFC…OLS_SORT (SYNSC-730point4) | 37 of 56 [63/514907] NFC…LS_INDEX (SYNSC-742point1) | 36 of 37 [4a/82f621] NFC…XTRACTOR (SYNSC-742point4) | 10 of 56 [11/3977b9] NFC…CYTOSINE (SYNSC-742point4) | 8 of 10 [b8/7580fa] NFC…K_REPORT (SYNSC-739point2) | 9 of 10 [- ] NFC…UP_BISMARK:BISMARK_SUMMARY - [- ] NFC…QMETHYLSEQQUALIMAP_BAMQC | 0 of 37 [- ] NFC…ETHYLSEQMETHYLSEQMULTIQC -

The Nextflow syntax/language server is complaining about calling

meta

in prefix and args etc... in our

modules.config

Any suggestions how else to use these variables within in the

modules.config

file 'properly'?

Ah no wait I just need to wrap the whole thing in a closure, ingnore me

Hi everyone! I'm trying to run scRNA-seq on a dataset processed with inDropsv2, and have set up everything as directed by the documentation at nf-core. However, when I try to run the pipeline, I"m getting the following error: WARN: Failed to render execution report -- see the log file for details WARN: Failed to render execution timeline -- see the log file for details ERROR ~ Access Denied (Service: S3, Status Code: 403, Request ID: 79QGY0A2V62NX5WJ, Extended Request ID: XlL+AoLMLEjJYbG2oOP8gphZ+fC8ULP+TigvxjqokXMU5ZJmpZ4kVA0XJeuTijMZryP7bFaq9j49jQ2sN+cKlA==) (SDK Attempt Count: 1) -- Check script 'assets/nf-core/scrnaseq/subworkflows/nf-core/utils_nextflow_pipeline/main.nf' at line: 80 or see '.nextflow.log' file for more details Given the error, I'm fairly sure the error is due to an inability of the process to access my s3. I'm positive I've copied my aws access keys correctly into the mmc.config file, as I'm running the pipeline on a persistent headnode through MemVerge. I'm still relatively new to nextflow, and this is my first time running an nf-core pipeline, so it's entirely possible I'm missing something basic, but any help would be appreciated!

Hi there, I'm running the

circRNA

pipeline using 16 core and 320 gigs of RAM, everything was running fine until I reached the process `Process `NFCORE_CIRCRNACIRCRNABSJ_DETECTION:COMBINE_SAMPLES (all)`` terminated with error exit status (137):

Command executed [/home/z5628486/.nextflow/assets/nf-core/circrna/modules/local/combinebeds/filter/templates/filter.py]:
Command exit status:
  137

Command output:
  (empty)

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  /opt/conda/lib/python3.12/site-packages/upsetplot/data.py:303: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
    df.fillna(False, inplace=True)
  .command.run: line 168:    34 Killed                  /usr/bin/env python .command.sh
Container:
  /srv/scratch/work/singularity/community.wave.seqera.io-library-pandas_polars_pyarrow_upsetplot-8840b96e156438fc.img

it uses the whole 320gb of RAM; I tried putting limiter in the resourcelimits.config such as:

process {
    resourceLimits = [
        cpus: 16,
        memory: '300.GB',
        time: '23.h'
    ]
}

but the process ignores the config file and as such gets OOM killed again. Any suggestion would be helpful, thanks!

Hi! I'm about to create a new release for nf-core/pixelator, but GitHub says no, any idea what I'm doing wrong?

Hi all - is it possible to provide args for a given process at the workflow level? i.e. workflow level injection to overwrite module level

def args = task.ext.args ?: ''

for example, something similar to:

include { RASUSA as LONG_RASUSA  } from '../../../modules/nf-core/rasusa'

workflow DOWNSAMPLING {

    take:
    long_reads_ch // channel: [ val(meta), long_reads ]

    main:

    LONG_RASUSA (
            long_reads_ch,
            100,
            args: "--seed 1"
        )

or

include { RASUSA as LONG_RASUSA  } from '../../../modules/nf-core/rasusa'

def long_rasusa.ext.args = "--seed 1"

workflow DOWNSAMPLING {

    take:
    long_reads_ch // channel: [ val(meta), long_reads ]

    main:

    LONG_RASUSA (
            long_reads_ch,
            100
        )

I want to avoid using

conf/modules.config

or modifying/patching the underlying module

This message was deleted.

ERROR ~ Error executing process > 'NFCORE_CUTANDRUNCUTANDRUNPREPARE_GENOME:GUNZIP_GTF' Caused by: Failed to pull singularity image command: singularity pull --name depot.galaxyproject.org-singularity-ubuntu-20.04.img.pulling.1762379086803 https://depot.galaxyproject.org/singularity/ubuntu:20.04 > /dev/null status : 255 message: ERROR: pull is only supported for shub URIs i keep getting this error, does anybody know what the issue could be? Also fyi - i an running this on HPC which doesnt have docker installed so i had to create a conda environment and install singularity. However i keep getting this issue.

Hi all I am using nf-core proteinfold , I setting use_gpu=true but get error in multiqc step, in case I would like avoid to use gpu in multiqc step how can i do ?

Hello, I'm running nextflow run nf-core/rnadnavar -r 1.3.1 but it shows "Cannot find revision 1.3.1 --Make sure that it exists in the remote repository https:... . I want to run variant calling from rna-seq, anyone pls help.

Hi all - I try to run nf-core/hlatyping:

Hi all - I try to use the pipeline nf-core/hlatyping on a computer with 4 CPUs and 64GB RAM. The bam file is 65 GB big and aligns 150 nucleotides reads paired-end. I got as full message error:

Error executing process > 'NFCORE_HLATYPING:HLATYPING:CHECK_PAIRED (BAPA1)'

Caused by:
  Process `NFCORE_HLATYPING:HLATYPING:CHECK_PAIRED (BAPA1)` terminated with an error exit status (127)


Command executed:

  if [ $({ samtools view -H pbmc-1_OE0224_BEHINDMS_BAPA1_merged.mdup.bam -@2 ; samtools view pbmc-1_OE0224_BEHINDMS_BAPA1_merged.mdup.bam -@2 | head -n1000; } | samtools view -c -f 1  -@2 ) -gt 0 ]; then
      echo false > is_singleend.txt
  else
      echo true > is_singleend.txt
  fi
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_HLATYPING:HLATYPING:CHECK_PAIRED":
      samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//')
  END_VERSIONS

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.run: line 306: docker: command not found

Work dir:
  /Users/remy/work/38/04a4e23976be5292d1b39e7fb7c8e6

Container:
  <http://quay.io/biocontainers/samtools:1.16.1--h6899075_0|quay.io/biocontainers/samtools:1.16.1--h6899075_0>

Do you have an idea what is going wrong?

If I put a profile into nf-core/configs, that defines more profiles, are those "inner" profiles available when I call the pipeline? I.e. can I do

nextflow run nf-core/pipeline -profile instution,sub_profile

where

sub_profile

is defined in

institution

?

Hey all, I'm having reports from some users where a local config file at ~/.nextflow/config seems to be taking precedence over our institution profile on the nf-core repo passed with the

-profile

flag. Specifically, the profile passed uses singularity and not conda, but their local config does, and their run now looks to be using conda despite all the logs suggesting it read the profile correctly. I expect it's that I've missed something in the docs and need to update guidance internally, but I can't figure out how to have the specified profile override a local config. Thanks!

Does anyone have any suggestions to on how to track down the cause of a ConcurrentModificationException error within a process?

ERROR ~ Error executing process > 'NFCORE_CREATETAXDB:CREATETAXDB:PREPROCESSING:FIND_CONCATENATE_DNA (1)'

Caused by:
  java.util.ConcurrentModificationException

OK correction it's happening in multiple processes 😕