https://nf-co.re logo
Join Slack
Powered by
# createtaxdb
  • j

    James Fellows Yates

    11/14/2025, 10:15 AM
    <!channel> v2 of #C062H82UCRG has just been released! This is mostly fixing a bunch of sort of major bugs related to Kraken2/Bracken and DIAMOND, but also includes support for building of databases for two more tools - sylph and sourmash, and documents how to build databases that have just a single FASTA. An important deprecation is the removal of the automatic building of database
    tar
    archives due to a range of incompatibilities between kraken/bracken vs nextflow vs containers, but I don't think it will be a huge loss - and I also wrote documentation how to do it manually in the FAQ :) Otherwise thanks to @novelvirus @moritz @Sofia Stamouli '@ellmagu' for the help and contributions to this one! CHANGELOG: ###
    Added
    • #108 - Document workaround for building databases from single FASTA files, e.g. from NCBI RefSeq (by @pcantalupo) • #111 - Added sourmash reference building both for genomes and proteomes (by @Midnighter). • #117 - Updated to nf-core/tools template 3.4.1 (by @jfy133) • #124 - Added sylph reference building (by @jfy133 and @sofstam). ###
    Fixed
    • #110 - Corrected the documented structures of the grouped output from the
    PREPROCESSING
    subworkflow (by @Midnighter). • #121 - Fix Kraken2 build failing with local files due to symlink-in-symlink mounting error with containers (❤️ to @ellmagu for reporting, fix by @jfy133) • #122 - Update DIAMOND to support more recent versions of NCBI taxonomy (by @jfy133) • #123 - Fix a MALT build validation check incorrectly assigned to --build_krakenuniq (by @jfy133) • #133 - Fix Kaiju compatible renamed FASTA files being always published even if --kaiju_keepintermediates false (by @jfy133) ###
    Dependencies
    | Tool | Old Version | New Version | | -------- | ----------- | ----------- | | sourmash | | 4.9.4 | | sylph | | 0.9.0 | | kraken2 | 2.1.5 | 2.1.6 | | tar | 1.34 | | | nf-core | 3.3.2 | 3.4.1 | | DIAMOND | 2.1.12 | 2.1.16 | | MultiQC | 1.31 | 1.32 | | Nextflow | 24.04.2 | 25.04.2 | ###
    Deprecated
    • #118 - Deprecated automated building of
    tar.gz
    archives of all databases (by @jfy133) • #126 - Remove guidance about needed double and single quotes when giving to
    --<tool>_build_options
    as actually not necessary (by @jfy133)
    🎉 3
    🙌 2
    👍🏽 1
    dancing penguin 1
  • j

    Júlia Mir

    11/19/2025, 4:08 PM
    Hello, I have triggered a template sync PR to this pipeline, if someone is available could you try to resolve conflicts and do the template update to report back any possible difficulty you might find?
    j
    • 2
    • 10
  • a

    Alexander Ramos Díaz

    11/28/2025, 2:46 AM
    Hello, is anyone getting a similar error when running the pipeline?:
    Copy code
    ERROR ~ Error executing process > 'NFCORE_CREATETAXDB:CREATETAXDB:PREPROCESSING:FIND_CONCATENATE_DNA (1)'
    
    Caused by:
      java.util.ConcurrentModificationException
    
     -- Check '.nextflow.log' file for details
    I didn't change anything it's just the main branch (
    nextflow run . -profile test,docker --outdir test_docker
    ). Nextflow 25.10.0 build 10289.
  • j

    James Fellows Yates

    11/28/2025, 5:42 AM
    Yes, it's happening everywhere (but not regularly) https://github.com/nf-core/createtaxdb/issues/114 :( but it's very hard to debug
    😮 1
  • j

    James Fellows Yates

    12/08/2025, 3:26 PM
    @Alexander Ramos Díaz thank you very much for the PR for metacsche! I'll get to reviewing it some point after the metaomics exhibition day on Wednesday :) (probably next week when I can finally get back to some coding)
    ✔️ 1
    a
    • 2
    • 7
  • j

    James Fellows Yates

    01/06/2026, 10:38 AM
    Would anyone be available to test drive this new tutorial?: https://github.com/nf-core/createtaxdb/pull/144 For me it only takes about 5-10 minutes (mostly from downloading large files)
    • 1
    • 2
  • j

    James Fellows Yates

    02/10/2026, 2:18 PM
    <!channel> v2.1.0 Gracious Goblin is now out! Thanks to @Florent Lassalle @Alexander Ramos Díaz @Diego Alvarez Saravia and @ChillarAnand for the help on this modest release! The most exciting thing is the addition of support for metacache! CHANGELOG: ###
    Added
    • #117 - Updated to nf-core/tools template 3.5.1 (by @jfy133) • #143 - Documented how to resovle KrakenUniq unbound variable jellyfish issue (❤️ to @flass for suggesting, added by @jfy133) • #140 - Added MetaCache database building support (❤️ to @ChillarAnand for suggestion, added by @alxndrdiaz and @jfy133) • #144 - Added tutorial on how to convert an NCBI
    assembly_summary.txt
    file to input samplesheet (❤️ to @dialvarezs for improvements, by @jfy133) ###
    Fixed
    ###
    Dependencies
    | Tool | Old Version | New Version | | --------- | ----------- | ----------- | | nf-core | 3.4.1 | 3.5.1 | | MetaCache | | 2.5.0 |
    badgerbadger 1
    🙌 1
  • j

    James Fellows Yates

    04/10/2026, 7:09 AM
    PR ready for review: https://github.com/nf-core/createtaxdb/pull/160
    s
    • 2
    • 2
  • j

    James Fellows Yates

    04/27/2026, 12:19 PM
    Anyone around for another small-medium sized review: https://github.com/nf-core/createtaxdb/pull/182 ? Seems like a lot of files (22), but it's mostly module updates, and updating tests for a local subworkflow. • Updates KMCP to correctly emit correct taxonomy files (with docs updates) • Updates KMCP subworkflow based on the above • Removes the now no-longer-needed workaround parameters added in #165 • Adds taxonomy files to test_minimal.config so the test config runs out of the box • Improves ordering of modules.config
    ✅ 1
  • j

    James Fellows Yates

    04/27/2026, 12:20 PM
    Once this is in I can prepare for a release!
  • f

    Full test notifications

    04/30/2026, 1:14 PM
    ✅ createtaxdb/test_full completed successfully! 🎉 Run Name festering_avogadro Duration 21m 15s Status ✅ Success Tasks Completed: 20 Apr 30, 2026 at 1:14 PM 🔗 View in Seqera Platform button
    catjam 1
    🎉 4
  • j

    James Fellows Yates

    04/30/2026, 1:14 PM
    phew
  • j

    James Fellows Yates

    05/11/2026, 3:28 PM
    Cross-posting a question from @Mahesh Binzer-Panchal: https://nfcore.slack.com/archives/C031QH57DSS/p1778502230592529
  • j

    James Fellows Yates

    05/11/2026, 3:32 PM
    And short answer: in #C062H82UCRG it was indeed implemented with a list of files: https://github.com/nf-core/createtaxdb/blob/e561e64257492bb337a4ade1555ecb772156a0c2/workflows/createtaxdb.nf#L173-L184 rather than a directory. So
    meta.yaml
    is wrong. I guess this is typical 'forget to update docs' after changing the strategy as I was ahving issues with symlinks in containers.
  • j

    James Fellows Yates

    05/11/2026, 3:33 PM
    It's challenging because some tools require a strict and complicated structure so I had early on tried to prefer directories rather than 'reconstructing' this directory
  • j

    James Fellows Yates

    05/11/2026, 3:33 PM
    but then it became more problematic
  • j

    James Fellows Yates

    05/11/2026, 3:33 PM
    I will try and make a PR now if I have time to fix the meta
  • j

    James Fellows Yates

    05/11/2026, 3:42 PM
    @Mahesh Binzer-Panchal here is a docs update: https://github.com/nf-core/modules/pull/11587 Ran out of time to look into fixing the 'clean intermediates' bit, but I guess just the find command could be refined to not delete those. As a side note, I think maybe the other reason why it's a bit ugly is because I think Krakenuniq will autodownload taxdmp files for you if you don't supply them... so maybe taht's why the description was a bit odd (at first I just assumed someone could supply the ame thing downloaded outside, but I can't remember now)
    approved with comments 1
  • m

    Mahesh Binzer-Panchal

    05/12/2026, 7:01 AM
    I'm working on the clean intermediates. At this point I'm leaning to copying the input rather than mv'ing and finding, but I'm still checking the tests. I've updated the taxonomy files to be the files rather than the directories
  • j

    James Fellows Yates

    05/12/2026, 7:11 AM
    Copying all inputs? Or just the taxonomy files?
    m
    • 2
    • 8
  • m

    Mahesh Binzer-Panchal

    05/12/2026, 10:56 AM
    Fix for the input deletion with Krakenuniq/build: https://github.com/nf-core/modules/pull/11607
    j
    • 2
    • 19
  • j

    James Fellows Yates

    06/04/2026, 9:42 AM
    @Haris Spyridis I EVENTUALLY reviewed your PR 🙃
  • c

    Charles Xu

    07/03/2026, 1:06 AM
    Is it reasonable for
    --build_kraken2
    on 7 eukaryotic genomes to take over 2 hours? I'd like to add all archaea and bacteria to this for host removal, but I must be doing something wrong...
    KRAKEN2_ADD
    is not parallelizing but I guess that is normal for reading the ~14GB
    nucl_gb.accession2taxid
    ?
    j
    • 2
    • 15
  • f

    Full test notifications

    07/03/2026, 8:20 AM
    ✅ createtaxdb/test_full completed successfully! 🎉 Run Name curious_plateau Duration 33m 49s Status ✅ Success Tasks Completed: 21 Jul 3, 2026 at 8:20 AM
  • j

    James Fellows Yates

    07/03/2026, 8:34 AM
    Smallish release (for @Charles Xu), but includes nice new functionality! v3.1.0 - Seductive Siren - brings the pipeline up to speed with the latest nf-core template, and adds a new building tool for Centrifuger as well as improved KRakenUniq cleanup thanks to @Haris Spyridis and @Mahesh Binzer-Panchal respectively! ###
    Added
    • #194 - Updated to nf-core template 4.0.2 (by @jfy133) • #195 - Added Centrifuger database building support (by @haris18s) ###
    Fixed
    • #196 - Improved KrakenUniq build clean up step (by @mahesh-panchal, @jfy133) ###
    Dependencies
    | Tool | Old Version | New Version | | ---------------- | ----------- | ----------- | | nf-core template | 4.0.1 | 4.0.2 | | nf-schema | 2.5.1 | 2.7.2 | | Centrifuger | | 1.1.0 |
  • f

    Full test notifications

    07/03/2026, 8:34 AM
    ✅ createtaxdb/test_full completed successfully! 🎉 Run Name elated_montalcini Duration 35m 5s Status ✅ Success Tasks Completed: 21 Jul 3, 2026 at 8:34 AM
  • f

    Full test notifications

    07/03/2026, 9:02 AM
    ✅ createtaxdb/test_full completed successfully! 🎉 Run Name crazy_raman Duration 32m 48s Status ✅ Success Tasks Completed: 21 Jul 3, 2026 at 9:02 AM
  • j

    James Fellows Yates

    07/24/2026, 6:17 AM
    Would anyone be available for a review of this medium sized PR? Slightly more taxing but it's quite important - as it fixes those stupid 'ConcurrentModificationErrors'!!!! https://github.com/nf-core/createtaxdb/pull/202/changes 1. It switches groupTuple to the new groupBy at the suggestion of @Ben Sherman (<3) to fix the error 2. Refactors the building steps into a dedicated subworkflow, which I wanted to do but also needed to do to get the nf-tests to trigger properly on GHA for some reason
    😲 1
    m
    • 2
    • 2
  • h

    Haris Spyridis

    09/07/2026, 4:05 PM
    Hello everyone, Question while adding sourmash to taxprofiler. The
    nf-core/modules/sourmash/sketch
    module hardcodes
    --merge
    , so in createtaxdb all reference genomes get merged into ONE signature per DB (confirmed with
    sourmash sig describe
    ). Sourmash GATHER would return only 1 match, since db has only one merge signature. and then tax annotate cant map it to species... We probably need per-genome signatures (via
    --singleton.
    (Note: i tried to use createtaxdb, so can generate some test files for taxprofiler). Any thoughts?
    j
    m
    • 3
    • 37
  • f

    Full test notifications

    10/08/2026, 8:09 AM
    ❌ createtaxdb/test_full failed 😿 Run Name friendly_monod Duration 17.9s Status ❌ Failed Error Message
    Copy code
    Missing process or function paramsSummaryLog([[:], repository: <https://github.com/nf-core/createtaxdb>, projectDir: /.nextflow/assets/.repos/nf-core/createtaxdb/clones/91f943c85605844c25eef904b10428ff3a09f124, commitId: 91f943c85605844c25eef904b10428ff3a09f124, revision: dev, startTime: 2026-10-08T08:09:38.556872355Z, endTime: null, duration: null, container: {}, commandLine: nextflow run '<https://github.com/nf-core/createtaxdb>' -name friendly_monod -params-file '<https://api.cloud.seqera.io/ephemeral/1AepL9yksblCPkMIEkGYDg.json>' -with-tower -r dev -profile test_full, nextflow: nextflow.NextflowMeta(version:26.04.6, build:12646, timestamp:09-07-2026 18:49 UTC, preview:nextflow.NextflowMeta$Preview@624baf71, enable:nextflow.NextflowMeta$Features@70799ca7, strictModeEnabled:false, moduleBinariesEnabled:false), success: false, workDir: /nf-core-awsmegatests/work/createtaxdb/work-dev, launchDir: /, profile: test_full]) -- Did you mean 'paramsSummaryLog' instead?
    Command Line
    Copy code
    nextflow run '<https://github.com/nf-core/createtaxdb>' -name friendly_monod -params-file '<https://api.cloud.seqera.io/ephemeral/1AepL9yksblCPkMIEkGYDg.json>' -with-tower -r dev -profile test_full
    Oct 8, 2026 at 8:09 AM