James Fellows Yates
02/10/2025, 9:37 AMJim Downie
02/10/2025, 9:40 AMMartin Beracochea
02/13/2025, 10:40 AMJim Downie
02/13/2025, 10:46 AMMartin Beracochea
02/13/2025, 2:53 PMHere is what I would try to do:
Instead of recreating the GTDB-Tk database I would download it and strip the data you don't need.
To reduce the Memory usage:
I would run "classify_wf" only with Archaeal genomes, this is the smaller tree we have . ( in the release220/pplacer/gtdb_r220_ar53.refpkg/
I would pick test genomes I know would be placed in a "small" phylum ( i.e. a phylum with few species like p__Undinarchaeota which has 6 species).
I would strip the Archaeal Tree used in Tk from all species not in p__Undinarchaeota EXCEPT for 1 per phylum ( This helps to keep a decent tree topology for pplacer)
I would parse the MSA used for pplacer to only keep species in your now trimmed Archaea tree.
The size of the new tree would be going from 5,869 species to only 26 species ( 6 for p__Undinarchaeota and 20 for the remaining 20 phyla) so the memory used by pplacer should be less than 16Gb.
You will need to recreate the pplacer package with this new tree + msa to regenerate the fitting_stats file ( you can find a basic guide under https://github.com/Ecogenomics/GTDBTk/wiki/Prepare-GTDB-Tk-data 4 - Create pplacer Package:
You can now submit one of the test genomes ( that has to be part of p__Undinarchaeota) with classify_wf ( with the option –skip_ani_screen) to the new GTDB-Tk. GTDB_Tk will pick it as an archaea, will place it in the new trimmed archaeal tree with pplacer under "p__Undinarchaeota" which is the only complete phylum. It will then run an ANI against only species in this phylum ( maximum 6)
To reduce storage used:
Because the ANI will only happen against 6 genome you can create empty genome files ( or maybe remove) for all genomes in the release220/skani/database folder EXCEPT for the 6 reps of p__Undinarchaeota (GB_GCA_002502135.1, GB_GCA_002495465.1, GB_GCA_002687935.1, GB_GCA_014189685.1, GB_GCA_014189675.1, GB_GCA_027077325.1)
Again this is purely theoretical but I think it will work
I haven't tried this yetJames Fellows Yates
02/13/2025, 2:57 PMMartin Beracochea
02/13/2025, 2:57 PMMartin Beracochea
02/13/2025, 2:57 PMJames Fellows Yates
02/13/2025, 2:57 PMJames Fellows Yates
02/13/2025, 2:58 PMclassify_wf) how is that recreating the database?Martin Beracochea
02/13/2025, 3:03 PMJames Fellows Yates
02/13/2025, 3:13 PMMartin Beracochea
02/13/2025, 3:15 PMJim Downie
02/13/2025, 3:28 PM.fna files of the whole genome for every single genome in GTDB. So I think this seems reasonable - filter down to a sensible subset of phyla, trim the trees, and then you can bin most of those genome files which constitute most of the space on disk.
But I don't like the idea of doing it for Archaea given that bacteria are much more common (e.g. the mag test data has bacteria)Martin Beracochea
02/13/2025, 3:30 PM