Its the first time that I use nf-core and Im trying to run the pipeline in the ifb cluster . Im getting an error but I dont understand what is happening. My core:

#!/bin/bash
#
#SBATCH -o logs/nfcoremetatdenovo.log_%J.out
#SBATCH -e logs/nfcoremetatdenovo.log_%J.err
#SBATCH --mail-type END
#SBATCH --mail-user <mailto:carla.pereira-garcia@univ-littoral.fr|carla.pereira-garcia@univ-littoral.fr>
#SBATCH --cpus-per-task 24
#SBATCH --mem 150GB
#SBATCH -p long

cd /shared/projects/margo_bac/metaT/nf-core-metatdenovo
module purge
module load nextflow
module load singularity
module load openjdk

nextflow run nf-core/metatdenovo \
	-profile ifb_core \
	-r 1.2.0 \
	--input /shared/projects/margo_bac/metaT/information/samplesheet_v2.csv \
	--outdir /shared/projects/margo_bac/metaT/output/11_metatdenovo \
	--sequence_filter /shared/projects/margo_bac/metaT/dbs/databases_sortRNA/SILVA_138_2_LSSU_tax_silva.fasta \
	--assembler spades \
	--min_contig_length 500 \
	--orf_caller prokka \
	--kofam_dir /shared/projects/margo_bac/dbs/KEGG_db/ \
	--hmmdir /shared/projects/margo_bac/dbs/FeGenie/iron/ \
	--skip_eggnog \
	--eukulele_method mets \
	--eukulele_db phylodb

And this is the log file

I actually don't see what the error can be in the log file. The closest I come is

WARNING: While bind mounting '/shared:/shared': destination is already in the mount point list

. This doesn't seem to be an error though. I think you need to look into the work directory for the process that failed (

/shared/ifbstor1/projects/margo_bac/metaT/nf-core-metatdenovo/work/7d/4f3df2ef08603d9ef8bef3f34fc1ad

). Check the content of the

.command.log

and

.command.err

files and see if that helps you. Don't hesitate to post further questions.

Thank you for your answer, I found my error and try to send it again. I’m still waiting for the results but I saw that in the .out file it says metatdenovo:spades | 2 of 2, failed 1, retries 1 I thought the pipeline did individual assemblies for all the samples, and it looks like it merged all and did one assembly. Can I change that??

No, you can't change that since it's the whole idea of the pipeline. It does one single assembly over all samples to arrive at a single set of ORFs that can then be quantified for the different samples so you can compare expression levels between samples.

But in the feature counts results I only have one sample. It finished already and I can see that it did 11 bbmaps (all my samples) but only one samtools sort, samtool index and feature counts. Why is that?

Sounds strange. If the pipeline failed somewhere it shouldn't output anything for featureCounts. And the

.nextflow.log

you sent shows it failed already at the

BBDUK

step which is much before assembly, ORF calling and quantification. When you run the pipeline next time, try to run the Nextflow process from the login/head node so you can monitor its progress more easily.

Yes it did failed the first time because I had added the wrong path to the ribosomal database but I fixed that and it run almost all the steps. It failed at the end in the eukulele clasification.

Then you should have all samples quantified. Check the

<outdir>/summary_tables/*.counts.tsv.gz

file.

I don’t have that table. In the summary_tables folder I only have

spades.prokka.kofamscan.tsv

and that table has prokka identifiers but it does not have the prokka annotations that I normally have, no ec number, no cog and no description; only kegg information

Are you sure nothing else failed? Or is maybe the pipeline not done yet? (The

spades.prokka.kofamscan.tsv.gz

is reformatted output from KOfamScan. If you're using Prokka as the ORF caller, the ORFs in will have Prokka identifiers. You should, eventually, also get a reformatted file with Prokka annotations in the

summary_tables

directory.)

I don’t find it. In the .out file this is the last part:

executor >  slurm (309)
[49/a99d27] NFC…TDENOVO:CAT_FASTQ (Fe-100) | 11 of 11 ✔
[3a/3ae9a1] NFC…_TRIMGALORE:FASTQC (Fe-32) | 11 of 11 ✔
[fc/3b8d89] NFC…MGALORE:TRIMGALORE (Fe-10) | 11 of 11 ✔
[be/4ede48] NFC…DENOVO:BBMAP_BBDUK (ADN-0) | 11 of 11 ✔
[06/00d6ec] NFC…NOVO:SEQTK_MERGEPE (Fe-10) | 11 of 11 ✔
[b0/65ce4e] NFC…TESPADESYAML (spades.yaml) | 1 of 1 ✔
[0c/8aa049] NFC…ETATDENOVO:SPADES (spades) | 2 of 2, failed: 1, retries: 1 ✔
[86/60fe5a] NFC…NOVO:FORMATSPADES (spades) | 1 of 1 ✔
[70/e6fa4a] NFC…SEQ_CONTIG_FILTER (spades) | 1 of 1 ✔
[fd/0d8496] NFC…ROKKA (spades.seqtk-seq.2) | 11 of 11 ✔
[eb/eb6dcb] NFC…TS:GFF_CAT (spades.prokka) | 1 of 1 ✔
[60/71943e] NFC…TS:FAA_CAT (spades.prokka) | 1 of 1 ✔
[05/a7071c] NFC…TS:FFN_CAT (spades.prokka) | 1 of 1 ✔
[-        ] NFC…OKKA_SUBSETS:PROKKAGFF2TSV | 0 of 1
[ed/934d33] NFC…UNPIGZ_GFF (prokka.gff.gz) | 1 of 1 ✔
[be/998e54] NFC…spades.seqtk-seq.fasta.gz) | 1 of 1 ✔
[00/af6ea8] NFC…DENOVO:BBMAP_ALIGN (Fe-10) | 11 of 11 ✔
[99/4a9764] NFC…EARCH (spades.prokka.Cyc1) | 215 of 215 ✔
[-        ] NFC…DENOVO:HMMCLASSIFY:HMMRANK | 0 of 1
[-        ] NFC…MCLASSIFY:SEQTK_HMMHITFAAS -
[ef/85c3c4] NFC…LS:SAMTOOLS_SORT (ADN-100) | 1 of 1
[64/26ea74] NFC…S:SAMTOOLS_INDEX (ADN-100) | 1 of 1
[3a/98bb8b] NFC…EATURECOUNTS_CDS (ADN-100) | 1 of 1
[skipped  ] NFC…MSCAN_DOWNLOAD (KEGG data) | 1 of 1, stored: 1 ✔
[a0/638000] NFC…MSCAN_SCAN (spades.prokka) | 1 of 1 ✔
[05/8aa587] NFC…spades.seqtk-seq.fasta.gz) | 1 of 1 ✔
[0b/b45c41] NFC…TDENOVO:TRANSRATE (spades) | 1 of 1 ✔
[72/fbf29f] NFC…ELE_SEARCH (spades.prokka) | 1 of 1, failed: 1 ✘
Plus 17 more processes waiting for tasks…
-[nf-core/metatdenovo] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_METATDENOVO:METATDENOVO:SUB_EUKULELE:EUKULELE_SEARCH (spades.prokka)'

Caused by:
  Process `NFCORE_METATDENOVO:METATDENOVO:SUB_EUKULELE:EUKULELE_SEARCH (spades.prokka)` terminated with an error exit status (1)


Command executed:

  rc=0
  mkdir contigs
  gunzip -c prokka.faa.gz > ./contigs/proteins.faa
  EUKulele \
      -m mets \
      --database phylodb \
      --protein_extension .faa \
      --reference_dir eukulele \
      -o spades.prokka_phylodb \
      --CPUs 12 \
      -s \
      contigs || rc=$?
  
  gzip spades.prokka_phylodb/mets_full/diamond/*.out
  find spades.prokka_phylodb/ -name "*.csv" | xargs gzip
  gzip spades.prokka_phylodb/taxonomy_estimation/*.out
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_METATDENOVO:METATDENOVO:SUB_EUKULELE:EUKULELE_SEARCH":
      eukulele: $(echo $(EUKulele --version 2>&1) | sed -n 's/.* \([0-9]\+\.[0-9]\+\.[0-9]\+\).*/\1/p')
  END_VERSIONS
  
  if [ $rc -le 1 ]; then
      exit 0
  else
      exit $rc;
  fi

Command exit status:
  1

Command output:
  All reference files for PhyloDB downloaded to phylodb
  Running EUKulele with command line arguments, as no valid configuration file was provided.
  Setting things up...
  Could not successfully install all external dependent software.
  Check DIAMOND, BLAST, BUSCO, and TransDecoder installation.
  ['proteins']
  Specified reference directory, reference FASTA, and protein map/taxonomy table not found. Using database in location: eukulele/phylodb.
  Automatically downloading database phylodb . If you intended to use an existing database folder, be sure a reference FASTA, protein map, and taxonomy table are provided. Check the documentation for details.

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  mkdir: can't create directory 'eukulele/': No such file or directory
  mkdir: can't create directory 'eukulele/': No such file or directory
  eukulele/phylodb/reference.pep.fa.gz: No such file or directory
  gunzip: eukulele/phylodb/reference.pep.fa.gz: No such file or directory
  eukulele/phylodb/taxonomy-table.txt.gz: No such file or directory
  gunzip: eukulele/phylodb/taxonomy-table.txt.gz: No such file or directory
  gzip: spades.prokka_phylodb/mets_full/diamond/*.out: No such file or directory

Then it says to check the

.nextflow.log

but I don’t find where it failed other than in the eukulele step

First, the Eukulele error is something I fixed last week, but that's not yet released. If you create a directory called

eukulele

in the directory where you run the pipeline, or somewhere else and point to it with

--eukulele_dbpath <dir>

and then resume the pipeline, it should work.

Okey, I need to download the database for eukulele before then? Steps to fix it: mkdir eukulele ( in the folder where the pipeline is running) Download the database and specify the path Add -resume to the same script and sent it If I just say -resume it will pick up where it left off right? I don’t need to specify the sesion or anything?

No need to download if you want to use phylodb. At least with my testing it seems to work. Just create the directory and add

-resume

.

Great! Thank you so much! I’ll try this and hope for the best!

Let me know if it works!

I will, I’m having trouble downloading the gtdb from the figshare. Is there a trick to download it directly to the cluster?

I use

wget

, other people use

curl

. With the former use commands like:

wget -O gtdb.r220.names.dmp <https://figshare.scilifelab.se/ndownloader/files/52095806>

Note that FigShare have stupid names of urls (just numbers), so you need to specify the name you want for the file. It's just to go through the files you need and specify the name after

-O

(that's the letter, not the number).

Yeah, that the first thing I tried.

--2025-08-11 13:11:57--  https://figshare.scilifelab.se/ndownloader/files/52095806
Resolving figshare.scilifelab.se (figshare.scilifelab.se)... 34.246.112.34, 52.49.8.221, 54.77.128.13, ...
Connecting to figshare.scilifelab.se (figshare.scilifelab.se)|34.246.112.34|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2025-08-11 13:11:58 ERROR 403: Forbidden.

I think it might be some restriction from my cluster, I have asked the administrators.

Fails for me too. Downloading from the web browser works. Usually it works with command line clients. I suppose it's difficult for you to download from your lap-/desktop first and then transfer to the cluster is difficult, or?

I try that already in my first attempt to run the pipeline and it keep getting cancel… I will try from my work computer and see if I manage. Thank you anyway!

I only meant the files you need for the Diamond taxonomy annotation.

Yes, but one of them is like 100 gb (gtdb-r220.taxonomy.dmnd ) I have another question for you if you don’t mind. I have a folder

bbmap/bbduk

in my results that has fasta files but I don’t have the

*.bbduk.log

that is suppose to come with them. Are those fasta files the reads that mapped to the reference (rRNA) or the unmapped (mRNA)?

Yes, that file is very large, I'm afraid. I'll check with support why download from the command line doesn't work.

I manage to download and do rsync so the pipeline is running again! Thank you so much for your help!!

I have the logs. But I looked through what was saved just a little while ago and those changes are not yet released and I don't remember what was being saved in earlier versions of the pipeline. If you write a config file, e.g.

nextflow.config

in the root directory of your analysis with the below content, they should be saved. (Just

-resume

the pipeline, and they should be saved wo the processes actually being rerun.):

process {
    withName: BBMAP_BBDUK {
        publishDir = [
            [
                path : { "${params.outdir}/bbmap/bbduk/"},
                pattern: '*.bbduk.log',
                mode: params.publish_dir_mode
            ]
        ]
    }
}

There are different ways of feeding config files to the pipeline. The easiest, IMO, is to call it

nextflow.config

, in which case it will be automatically used by Nextflow. the second easiest is to pass it to Nextflow with

-c my.config

.

Thank you for that! And for everything really! You have been really helpful and I really like your pipeline!

Thank you for trying it out so we can make it better!

Hey one more question, just curious is it possible to specify two orf callers, both prodigal and prokka? I’m interested in specific iron genes and usually it finds more genes with prodigal but I kinda trust more the prokka results

Yes. You should be able to

--orf_caller prodigal,prokka

. If that doesn't work, tell me, and in the meanwhile just

-resume

the pipeline with the other. In

<outdir>/summary_tables

file names should be fine, but if you see overwriting of files elsewhere, let me know.

Great! Thanks!!

The FigShare files are downloadable with wget using urls looking like this: https://ndownloader.figshare.com/files/52095806 instead of the links on the web page.

Okey great! But the way, the pipeline fails when you give it more than one orf caller, it just said error validating options.

The following invalid input values have been detected:

* --orf_caller (prokka,prodigal): Expected any of [[prodigal, prokka, transdecoder]]

In case you were wondering.

Thanks! I'll try to fix that before the next release. In the meantime, if you just

-resume

specifying the other ORF caller it should work (as I already said...).