Hi, I am experiencing this error: WARN: [nf-core/t...
# taxprofiler
c
Hi, I am experiencing this error: WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input merge reads (e.g. with --shortread_qc_mergepairs) I have added this flag (--shortread_qc_mergepairs) to the run command, and I am still getting the same error. Any suggestions? Thank you! Note, I am on revision: 970bdc2155 [1.2.3]
j
Oooh that might be a bug...
Could you make a small reproducible example (with public data, or our test data) and make an issue?
But could also send your samplesheet and
.nextflow.log
?
Because teh code looks good 😅
c
Hi, thanks for the quick response. Here's the sample sheet and the top of my log file. Thanks again! [33mNextflow 25.04.6 is available - Please consider updating your version to it(B[m N E X T F L O W ~ version 25.04.3 Launching
<https://github.com/nf-core/taxprofiler>
[berserk_heisenberg] DSL2 - revision: 970bdc2155 [1.2.3] ------------------------------------------------------ ,--./,-. _ _ _ __ _ /,-._.--~' |\ | |__ _ / ` / \ |_) |__ } { | \| | \__, \__/ | \ |___ \`-._,-
Copy code
-,
.,.,' nf-core/taxprofiler 1.2.3 ------------------------------------------------------ Input/output options input : /gpfs2/scratch/pi/msama/Spring_2025_short_reads/taxprofiler_07_2025/samplesheet.csv databases : /gpfs2/scratch/pi/msama/Spring_2025_short_reads/taxprofiler_07_2025/database.csv outdir : /gpfs2/scratch/pi/msama/Spring_2025_short_reads/taxprofiler_07_2025/output/ Preprocessing short-read QC options shortread_qc_mergepairs : true Profiling options run_diamond : true run_kraken2 : true run_bracken : true run_malt : true Postprocessing and visualisation options run_krona : true krona_taxonomy_directory: /gpfs2/scratch/pi/msama/Taxprofiler_db/krona/taxonomy/ Generic options trace_report_suffix : 2025-07-29_10-39-13 Core Nextflow options revision : 1.2.3 runName : berserk_heisenberg containerEngine : singularity launchDir : /gpfs2/scratch/pi/msama/Spring_2025_short_reads/taxprofiler_07_2025 workDir : /gpfs2/scratch/pi/msama/Spring_2025_short_reads/taxprofiler_07_2025/work projectDir : /users/c/t/ctilton/.nextflow/assets/nf-core/taxprofiler userName : ctilton profile : singularity configFiles : /users/c/t/ctilton/.nextflow/assets/nf-core/taxprofiler/nextflow.config, /gpfs2/scratch/pi/msama/Spring_2025_short_reads/taxprofiler_07_2025/malt_07_29.config !! Only displaying parameters that differ from the pipeline defaults !! ------------------------------------------------------ * The pipeline https://doi.org/10.1101/2023.10.20.563221 * The nf-core framework https://doi.org/10.1038/s41587-020-0439-x * Software dependencies https://github.com/nf-core/taxprofiler/blob/master/CITATIONS.md WARN: The following invalid input values have been detected: * --max_cpus: 20 * --max_memory: 200GB * --max_time: 48h [- ] NFC…PROFILERtaxprofilerUNTAR - [- ] NFC…ROFILERtaxprofilerFASTQC - [- ] NFC…PROFILERtaxprofilerUNTAR - [- ] NFC…ROFILERtaxprofilerFASTQC - [- ] NFC…ROFILERPROFILINGMALT_RUN - [- ] NFC…OFILING:MEGAN_RMA2INFO_TSV - [- ] NFC…PROFILINGKRAKEN2_KRAKEN2 - [- ] NFC…PROFILINGBRACKEN_BRACKEN - [- ] NFC…RPROFILINGDIAMOND_BLASTX - [- ] NFC…:KRAKENTOOLS_KREPORT2KRONA - [- ] NFC…ON_KRONA:KAIJU_KAIJU2KRONA - [- ] NFC…ZATION_KRONA:KRONA_CLEANUP - [- ] NFC…N_KRONA:KRONA_KTIMPORTTEXT - [- ] NFC…KRONA:MEGAN_RMA2INFO_KRONA - [- ] NFC…VISUALIZATION_KRONA:GUNZIP - [- ] NFC…ONA:KRONA_KTIMPORTTAXONOMY - [- ] NFC…PROFILERtaxprofilerUNTAR - [- ] NFC…ROFILERtaxprofilerFASTQC - [- ] NFC…ROFILERPROFILINGMALT_RUN - [- ] NFC…OFILING:MEGAN_RMA2INFO_TSV - [- ] NFC…PROFILINGKRAKEN2_KRAKEN2 - [- ] NFC…PROFILINGBRACKEN_BRACKEN - [- ] NFC…RPROFILINGDIAMOND_BLASTX - [- ] NFC…:KRAKENTOOLS_KREPORT2KRONA - [- ] NFC…ON_KRONA:KAIJU_KAIJU2KRONA - [- ] NFC…ZATION_KRONA:KRONA_CLEANUP - [- ] NFC…N_KRONA:KRONA_KTIMPORTTEXT - [- ] NFC…KRONA:MEGAN_RMA2INFO_KRONA - [- ] NFC…VISUALIZATION_KRONA:GUNZIP - [- ] NFC…ONA:KRONA_KTIMPORTTAXONOMY - [- ] NFC…OFILERtaxprofilerMULTIQC - WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample 1BK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample 1TK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample 3BK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample 3TK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample 4BK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample 4TK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample BBK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample BTK. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample neg. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample pos. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UI1. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UI2. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UI3. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UI4. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UI5. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UI6. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UIneg1. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample Undetermined. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UP1. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UP2. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UP3. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UP4. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UP5. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UP6. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UPbonus. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UPnegCT. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample UPposCT. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample US1. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample US2. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample US3. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample US4. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample US5. WARN: [nf-core/taxprofiler] DIAMOND does not accept paired-end files as input. To run DIAMOND on this sample, please merge reads (e.g. with --shortread_qc_mergepairs). Skipping DIAMOND for sample US6.
j
Can you send the actual
.nextflow.log
file`?
What i'm looking forward is somewhere a bit deeper
c
Sure! Apologies, I am new to this. This may be the one you are looking for. Let me know if you can't view it, and I will try to save it as a .txt.
j
That's the one!
Ah you used a
params.json
Do you mind sending taht too? (sorry 😅 )
Also I'm a conference so further suggestions may come later
Ah waiut I think I see the issue
--shortread_qc_mergepairs
only works if you give
--perform_shortread_qc
did you specify this?
It looks to me that in your log file your run didn't run FASTP or ADAPTERREMOVAL
c
No worries, here is the .json: { "input": "\/gpfs2\/scratch\/pi\/msama\/Spring_2025_short_reads\/taxprofiler_07_2025\/samplesheet.csv", "databases": "\/gpfs2\/scratch\/pi\/msama\/Spring_2025_short_reads\/taxprofiler_07_2025\/database.csv", "outdir": "\/gpfs2\/scratch\/pi\/msama\/Spring_2025_short_reads\/taxprofiler_07_2025\/output\/", "shortread_qc_mergepairs": true, "run_diamond": true, "run_kraken2": true, "run_bracken": true, "run_malt": true, "run_krona": true, "krona_taxonomy_directory": "\/gpfs2\/scratch\/pi\/msama\/Taxprofiler_db\/krona\/taxonomy\/" } I did also try including the shortread_qc_mergepairs directy in the command: nextflow run nf-core/taxprofiler \ -r 1.2.3 \ -profile singularity \ -work-dir /gpfs2/scratch/pi/msama/Spring_2025_short_reads/taxprofiler_07_29_2025/work/ \ -resume \ -params-file nf-params_07_29_2025.json \ -c malt_07_29.config \ --shortread_qc_mergepairs
j
(for whatever reason)
RIght yes - you didn't activate the shotread processing bit (which is within which the
--qc_mergepairs
is evaluated)
So try again but with adding
--perform_shortread_qc
c
Ah, okay! Thank you, I will try this!
👍 1
l
@Caleb Tilton any luck after including the parameter
--perform_shortread_qc
?
👀 1
s
I got same error I used both parameters but I get error in Kraken ' Process
NFCORE_TAXPROFILER:taxprofiler:PROFILING:KRAKENUNIQ_PRELOADEDKRAKENUNIQ
input file name collision -- There are multiple input files for each of the following file names:'
l
That’s weird. Can you check the
.command.sh
of this prcoess in
work
? Or can you add dump to the input channels of this process?
j
what does your input sheet look like?
s
unfortunately no folder created [- ] NFC…PROFILINGKRAKEN2_KRAKEN2 | 0 of 2 [e2/ae19a7] NFC…Diamond_refseq228|Control) | 2 of 2 ✔️ [- ] NFC…ENUNIQ_PRELOADEDKRAKENUNIQ - -[nf-core/taxprofiler] Pipeline completed with errors- ERROR ~ Error executing process > 'NFCORE_TAXPROFILERtaxprofilerPROFILING:KRAKENUNIQ_PRELOADEDKRAKENUNIQ (1)'
The input sheet is sample,run_accession,instrument_platform,fastq_1,fastq_2,fasta Sample,Run1,ILLUMINA,/STK/batch2_input_sequences/EBV/EBV_R1.fastq.gz,/STK/batch2_input_sequences/EBV/EBV_R2.fastq.gz, Control,Run1,ILLUMINA,/STK/input_sequences/human/human_R1.fastq.gz,/STK/input_sequences/human/human_R2.fastq.gz,
It worked with v1.1.8 , error after using 1.2.2
j
And your command/params.yml?
s
nextflow run nf-core/taxprofiler --perform_shortread_qc --shortread_qc_tool 'fastp' --shortread_qc_mergepairs --perform_shortread_hostremoval --shortread_hostremoval_index ${hostindex_c} --hostremoval_reference ${hostfasta_c} --save_preprocessed_reads --run_diamond --perform_shortread_complexityfilter --shortread_complexityfilter_tool 'bbduk' -profile singularity --input ${SampleSheet_c} --databases ${DatabaseSheet_c} --outdir TaxProfiler_out -r ${Taxver_c} -c ${MandaloreConfig_c} --run_kraken2 --kraken2_save_readclassifications --kraken2_save_reads --run_krakenuniq --krakenuniq_save_reads --krakenuniq_save_readclassifications --run_diamond --diamond_output_format tsv --run_krona --save_hostremoval_unmapped -resume
j
That is bizarre...
We build the names from
sample
and
run_accession
columns
but taht should be unique for you, as it should be just
sample_run1
and
sample_run_2
What do you have in
analysis_ready_fastqs
?
s
This is what I have in fastp folder Control_Run1_1.fastp.fastq.gz Control_Run1.fastp.log Sample_Run1.fastp.html Control_Run1_2.fastp.fastq.gz Control_Run1.merged.fastq.gz Sample_Run1.fastp.json Control_Run1.fastp.html Sample_Run1_1.fastp.fastq.gz Sample_Run1.fastp.log Control_Run1.fastp.json Sample_Run1_2.fastp.fastq.gz Sample_Run1.merged.fastq.gz
j
Ah sorry, could you try resuming but with
--save_analysis_ready_fastqs
?
s
ok
These are generated in analysis_ready_fastqs Control_Run1.unmapped.fastq.gz Sample_Run1.unmapped.fastq.gz
j
🤔 ok that's what I expect...
Can you send your
.nextflow.log
log?
file*
s
sorry that is wrong file, It shows error in diamond which was working before
j
Can you send the KU one?
s
I am rerunning I did so many trials and got confused with different folders
I will send it soon
Sorry, here is the right file
j
Oh wow we have a bug 😅
Copy code
Process `NFCORE_TAXPROFILER:TAXPROFILER:PROFILING:KRAKENUNIQ_PRELOADEDKRAKENUNIQ` input file name collision -- There are multiple input files for each of the following file names: sequences/b5, sequences/7d6e8ff80fd5dd7c9ebfa6dbba041b, sequences/Results, sequences/Test, sequences/STK, sequences/projects, sequences, sequences/Validation, sequences/work, sequences/logs, sequences/P23-015
That does not look right
I think it's trying to bring in possibly two databases at once?
What does your database TSV look like @Subazini TK?
s
tool,db_name,db_params,db_path kraken2,Kraken2_PlusPF_241228,,/databases/taxprofiler_validation/kraken2/k2_pluspf_20241228.tar.gz diamond,Diamond_refseq228,,/databases/taxprofiler_validation/Refseq_release_228/refseq_228_Bacteria_Virus_Archaea_Fungi_diamond.dmnd krakenuniq,microbialdb_minus_kdb_20230808,,/databases/taxprofiler/databases/Krakenuniq/microbialdb_minus_kdb_20230808
These are files inside Krakenuniq/microbialdb_minus_kdb_20230808 database100mers.kmer_distrib database50mers.kmer_distrib download.sh database125mers.kmer_distrib database75mers.kmer_distrib download.sh~ database150mers.kmer_distrib database.idx ktaxonomy.tsv database200mers.kmer_distrib database.kdb kuniq_microbialdb_minus_kdb.20230808.tgz database250mers.kmer_distrib database.kdb.counts seqid2taxid.map database300mers.kmer_distrib download.o595600 taxDB
l
To me, the database looks normal. @Subazini TK It seems multiple sequences are being copied into the same directory
sequences
in the working directory. Can you re-run analysis without
-resume
?
I ran test_full.config successfully this morning.
s
Thank you Lily, I will run and get back to you
It didnot work, getting same error
l
Do you mean the same error in Krakenuniq? I can look into that today
s
Thanks a lot, Yes the same error, It works when using 1.1.8 and throw this error when using 1.2.2. I use extra argument --qc_mergepairs in 1.2.2
j
We need to find out where the duplicates are coming from
OH WAIT I THINK I KNOW WHAT IT IS facepalm
l
Aha so you run 1.2.2? I tested version 1.2.3.
j
It's a single element list in Groovy that we are plucking something from but when it's a single element with
/
it splits the single path by
/
🙃
We need to found out where though
So this is a bug
@Subazini TK can you run a fresh run with a separate
work/
directory, and then do a
find -name 'P23-015'
or something, so we know what is the upstream place that duplicate comes from?
l
j
I assume it's from there OR it's from the modiule itself
And it can't handle a single sample for whatever reason
s
@James Fellows Yates I will rerun with new work dir and come back
j
❤️
s
If I add the flag --perform_runmerging, then it works without throwing error. I am running with --save_runmerged_reads to see what is merged?
j
Huh interesting...
Yes please also do that
Although that's odd... it should only be run-merging sample name... and you've only got one of each, I don't understand why that would work
Oooh unless the final channel construction misses a
[]
when we don't run merge?
s
Could you please tell me where exactly this [ ] could be possibly missed I can try to run adding it and see if it fixes the error without using --perform_runmerging
j
That's a code thingy but I don't know where exactly it should go
I would need to look (Lili might know) but I'm teaching this week
👍 1
l
Can I debug tomorrow morning? I am working with the clinical thing today.
nod 1
🙏 1
@Subazini TK Not sure why I always ran into the permission issue of version 1.2.2 today. I tested the latest version 1.2.3 instead and it worked fine.
wait, I missed
--krakenuniq_save_readclassifications
. Re-run it.
I am replicating the error by using the taxprofiler test data, not sure why this process
KRAKENUNIQ_PRELOADEDKRAKENUNIQ
was skipped, I will do a new test with the full test data:
Copy code
params {
    config_profile_name                    = 'Test profile'
    config_profile_description             = 'Minimal test dataset to check pipeline function'

    // Input data
    input                                  = params.pipelines_testdata_base_path + 'taxprofiler/samplesheet.csv'
    databases                              = params.pipelines_testdata_base_path + 'taxprofiler/database_v1.2.csv'
    save_preprocessed_reads                = true
    krakenuniq_save_readclassifications    = true
    save_hostremoval_unmapped              = true

    perform_shortread_qc                   = true
    perform_shortread_redundancyestimation = false
    perform_longread_qc                    = false
    shortread_qc_mergepairs                = true
    perform_shortread_complexityfilter     = true
    perform_shortread_hostremoval          = false
    perform_longread_hostremoval           = true
    perform_runmerging                     = false
    hostremoval_reference                  = params.pipelines_testdata_base_path + 'modules/data/genomics/homo_sapiens/genome/genome.fasta'
    run_kaiju                              = false
    run_kraken2                            = true
    run_bracken                            = false
    run_malt                               = false
    run_metaphlan                          = false
    run_centrifuge                         = false
    run_diamond                            = true
    run_krakenuniq                         = true
    run_motus                              = false
    run_ganon                              = false
    run_krona                              = true
    run_kmcp                               = false
    krona_taxonomy_directory               = params.pipelines_testdata_base_path + 'modules/data/genomics/sarscov2/metagenome/krona_taxonomy.tab'
    malt_save_reads                        = true
    kraken2_save_reads                     = true
    centrifuge_save_reads                  = true
    run_profile_standardisation            = true
}
j
@Lili Andersson-Li krakenuniqe has it's own profile 😉
l
yeah, right.
I replicated the error now, I am not sure if the error caused by this line https://github.com/nf-core/taxprofiler/blob/970bdc21556eb98135a0abe4766580b8d8556895/subworkflows/local/profiling.nf#L393 Here are the krakenuniq input reads channels:
Copy code
[DUMP: un-merged_reads] [['id':'db6', 'single_end':true, 'seqtype':'fasta', 'tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERX5474930_ERR5766174_1.fa.gz], ["<http://2611_ERR5766174.se|2611_ERR5766174.se>"]]
[DUMP: un-merged_reads] [['id':'db6', 'single_end':true, 'seqtype':'fasta', 'tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERR3201952_dup.fa.gz], ["<http://2614_ERR3201952.se|2614_ERR3201952.se>"]]
[DUMP: un-merged_reads] [['id':'db6', 'single_end':true, 'seqtype':'fastq', 'tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/99/bfe61fc37307e71b131d85aa2ee3ab/ERR3201952_ERR3201952_filtered.fastq.gz], ["<http://ERR3201952_ERR3201952.se|ERR3201952_ERR3201952.se>"]]
[DUMP: un-merged_reads] [['id':'db6', 'single_end':true, 'seqtype':'fastq', 'tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], [home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, 2b, 6cebfd0431953c617c07ec9af8491e, 2612_ERR5766176.unmapped.fastq.gz, home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, 01, 85309f465c7fb6c6fb476824c4e086, 2612_ERR5766176_B.unmapped.fastq.gz, home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, 9b, 8c4bf01c97a03dfadb0bab311da61e, 2612_ERR5766180.unmapped.fastq.gz, home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, 8a, 65b1bf18d5e2bc82433326abe50b47, 2613_ERR5766181.unmapped.fastq.gz], ["<http://2612_ERR5766176.se|2612_ERR5766176.se>", "<http://2612_ERR5766176_B.se|2612_ERR5766176_B.se>", "<http://2612_ERR5766180.se|2612_ERR5766180.se>", "<http://2613_ERR5766181.se|2613_ERR5766181.se>"]]
j
Yes, you will need to add a bunch of
dumps
around those modifications and see where the path gets split
l
Yes, I am testing now
❤️ 1
I found the error: https://github.com/nf-core/taxprofiler/blob/970bdc21556eb98135a0abe4766580b8d8556895/subworkflows/local/profiling.nf#L395 The read path is a single file path, not a list, which causes the issue for
reads + [prefix]
, we could convert reads into a list before adding prefix to the reads list:
Copy code
def readsList = reads instanceof List ? reads : [reads]
                    [[id: db_meta.db_name, single_end: meta.single_end, seqtype: seqtype], readsList + [prefix], db_meta, db]
I will create an issue and PR to fix that
So the error disappeared using
params.perform_runmerging
@Subazini TK you could using
--perform_runmerging
to avoid the issue for now.
j
Aha, that's indeed what I suspected... (at least along those lines)
So what
What I wonder is what does the channel look like whne you run with run merging... what is the difference?
l
Yes i saw your earlier message, you were right from the very beginning!!!
twitching eye 1
j
I wonder if we can wrap
[reads].flatten()
?
l
With run merging:
Copy code
[DUMP: perform_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fasta'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERX5474930_ERR5766174_1.fa.gz, 2611], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: perform_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fasta'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERR3201952_dup.fa.gz, 2614], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: perform_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/62/7799572abc3d5c6accc2366a628852/ERR3201952_ERR3201952.unmapped_other.fastq.gz, 'ERR3201952'], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: perform_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/b1/1970a403821c406a163e3b707260ed/2613_ERR5766181.unmapped.fastq.gz, 2613], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: perform_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/81/559825c8dd3dab395a92d8d08f3d8a/2612.merged.fastq.gz, 2612], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
without run merging after fixing the bug:
Copy code
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fasta'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERX5474930_ERR5766174_1.fa.gz, "2611_ERR5766174"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fasta'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERR3201952_dup.fa.gz, "2614_ERR3201952"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/62/7799572abc3d5c6accc2366a628852/ERR3201952_ERR3201952.unmapped_other.fastq.gz, "ERR3201952_ERR3201952"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/f9/f9c419b2fa854a6e6fc5627746f113/2612_ERR5766176_B.unmapped.fastq.gz, "2612_ERR5766176_B"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/42/6fbcd2051f7b615e7871a00d6f0923/2612_ERR5766180.unmapped.fastq.gz, "2612_ERR5766180"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/3a/7892755d50a2de3e9369dca0e33683/2612_ERR5766176.unmapped.fastq.gz, "2612_ERR5766176"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [/home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/b1/1970a403821c406a163e3b707260ed/2613_ERR5766181.unmapped.fastq.gz, "2613_ERR5766181"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
without run merging with the bug:
Copy code
UMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fasta'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERX5474930_ERR5766174_1.fa.gz, "2611_ERR5766174"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fasta'], [/nf-core/test-datasets/taxprofiler/data/fasta/ERR3201952_dup.fa.gz, "2614_ERR3201952"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, 62, 7799572abc3d5c6accc2366a628852, ERR3201952_ERR3201952.unmapped_other.fastq.gz, "ERR3201952_ERR3201952"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'long'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, 3a, 7892755d50a2de3e9369dca0e33683, 2612_ERR5766176.unmapped.fastq.gz, "2612_ERR5766176"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, b1, 1970a403821c406a163e3b707260ed, 2613_ERR5766181.unmapped.fastq.gz, "2613_ERR5766181"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, 42, 6fbcd2051f7b615e7871a00d6f0923, 2612_ERR5766180.unmapped.fastq.gz, "2612_ERR5766180"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
[DUMP: withou_runmerging] [['id':'db6', 'single_end':true, 'seqtype':'fastq'], [home, proj, development, microbial, metagenomics, lili, pipelines, nfcore, taxprofiler, work, f9, f9c419b2fa854a6e6fc5627746f113, 2612_ERR5766176_B.unmapped.fastq.gz, "2612_ERR5766176_B"], ['tool':'krakenuniq', 'db_name':'db6', 'db_params':'', 'type':'short'], /home/proj/development/microbial/metagenomics/lili/pipelines/nfcore/taxprofiler/work/08/8b23e21c149df8d067df0c2277a000/testdb-krakenuniq]
j
How did yo ufix the bug/
with the flatthen thingy?
l
Yes, with you smart suggestion:
Copy code
def readsList = [reads].flatten()
[[id: db_meta.db_name, single_end: meta.single_end, seqtype: seqtype], readsList + [prefix], db_meta, db]
and my way also works:
Copy code
def readsList = reads instanceof List ? reads : [reads]
[[id: db_meta.db_name, single_end: meta.single_end, seqtype: seqtype], readsList + [prefix], db_meta, db]
I could also combine these two lines:
Copy code
[[id: db_meta.db_name, single_end: meta.single_end, seqtype: seqtype], [reads].flatten() + [prefix], db_meta, db]
s
Thanks a lot @Lili Andersson-Li and @James Fellows Yates for fixing it quickly
l
Thank you for reporting the error! I will fix the bug in the next patch release.
🙏 1
j
OK nice!
We should triple check that all combinations of
reads
still works as expected (single, pairs, one set, multiple sets)
👍 1