Does anybody know of any "standard" short read dat...
# random
d
Does anybody know of any "standard" short read datasets that are commonly used in metagenomic benchmarking? I want to look at thread-scaling performance of Kraken2 on a few different systems.
j
There are things like the CAMI/2 datasets
or the Loman's mock community stuff: https://lomanlab.github.io/mockcommunity/
But CAMI2 is coming to be some of the most estalbished
d
Awesome. Many thanks for that info!
m
Didn't we want to look into that for the taxprofiler benchmarks on deNBI?
j
Yup!
d
You looked at Kraken2's thread scaling? Do you happen to recall what the results looked like? My initial runs on an Intel Xeon 8468 node using one of the CAMI2 data sets shows performance that is pretty different than what was reported in the Kraken2 paper. The overall performance is better, but the scaling is way worse than they reported. Kraken2 doesn't seem to scale past about 4-5 threads in my hands. I'm wondering if the performance is dependent on the size of either the input reads or the database.
m
We haven't yet, but had planned to do so systematically. Such benchmarks are also very dependent on implementation details.
As an example, did you use a memory mapped database? Otherwise for few samples, loading the database will easily dominate the time taken.
👍 1
d
Ah, good idea. I am using the Standard-16 index at the moment. It looks like loading the index does not make much of the total runtime, but I'll add the
--memory-mapping
flag to check.