Hi! Awesome job with this pipeline, I'm super exci...
# createtaxdb
z
Hi! Awesome job with this pipeline, I'm super excited to try it out. 🥳 I was wondering if you could give any rough size or runtime estimates across your database types (e.g. how many genomes went into the 9 minutes kraken-build run?). context: I have a 280 GB Kraken2 database (specifically for soil microbes, built with Struo2) and want to update with new MAGs. I'd love to build/share a Diamond database as well, but not sure if I will surpass my 450GB AWS Open Data allocation 😅
🎉 1
j
Hi @Zoey Werbin Thank you! We don't have a full information on that information yet, we may do a little bit of benchmarking for a paper. However honestly I'm never sure how representative things such as run time/resource usage is as it often varies based on a lot of things. That said for run time you can see this info: https://nf-co.re/createtaxdb/1.0.0/results/createtaxdb/results-08be5d25eb97f40f32a232e3ae12ef0d7f6c7fe5/pipeline_info/ (and attached -see the Job Duration graph) On AWS these were the basic run stats (see second screenshot) The number of genomes was 89: https://github.com/nf-core/test-datasets/blob/createtaxdb/samplesheets/samplesheet_test_full.csv
The final results folder of that particular run (which is all 9 classifiers, with tar archives of each database and modified Kaiju-compatible FASTAs) was 41GB And this gets extended to 49.8GB
Sorry doubled if you copy results from work to 90GB
But if you turn off a lot of options it'll be much smaller
DIAMOND resulting directory was 263.4MB Kraken2 was 7.1 GB (but this is without cleanup)
z
That's really helpful, thanks! I will also report back with any findings 🙂 (though it may be a while before I attempt)
❤️ 1
j
That would be really helpful, thank you!