<@UE6D8290F> was reading the maintainer's minutes....
# references
p
@Maxime Garcia was reading the maintainer's minutes. It says that no-one is using igenomes.config, but don't most pipelines use it? It's the backbone of
--genome
that's pretty commonly used, right?
m
most pipelines have it since it's part of template
The way I see it, the
igenomes.config
file is meant to be modified so that it match what you actually need in your pipeline
#CE8SSJV3N had a perfect usage for it, because that's why we created it at the beginning
But since #CELEMAGC9 was never updated, #CE8SSJV3N is not using it
#CGFUX04HZ is a good example of what I think people should do with the
igenomes.config
file
meaning, modify it
I know that @Sateesh_Peri has started recently to use it in #CFP55NZ5G
I'm pushing #C025UH66DA8 to use it
but more or less, most of the genomic pipelines mainly use params to directly access references
My main issue with this
igenomes.config
is that it's complex to update
Because no one (with minor exceptions) has an idea about what is actually in #CELEMAGC9
so the plan is 3 fold. 1 - update igenomes with #C073ZDLNC92 the pipeline (currently missing a few tools, but almost ready to go) That way we can already start updating
igenomes.config
if we want to. (but let's be realistic, I do think only a small amount of people will do this). 2 - provide an alternative to the
igenomes.config
that is yaml based, stored in a single separate repo (à la #CGRTMASKY in the idea) and contains all references stored in igenomes (plus additional values, similar to the
igenomes.config
in #CGFUX04HZ), with a schema, that can be versioned and can evolve depending on what has been updated on igenomes. The idea is to keep using the
--genome
key, with a slight modification (
GRCh38
->
Ensembl/GRCh38
or
GATK/GRCh38
). That way it's easier to get access to what is available for any genome. 3 - keep using this system for the reboot That way the transition is easy
p
Sounds sensible. Two thoughts / questions: 1. Can we have aliases so that
GRCh38
still works? (but log output etc could say the longer name) 2. Regarding the move from config to YAML - any downsides? I'm thinking particularly about: a. Being able to provide specific assets with params to overwrite what's set with
--genome
, for example something like
--genome CRCh38 --blacklist mybedfile.bed
(or whatever). Maybe still works if the various channels are still set using params that get defaults from the YAML? 🤔 b. Future parameter typing stuff from Nextflow (maybe also as above).
m
1. we could deal with that at the pipeline level, my main issue with GRCh38, is that in rnaseq it's Ensembl/GRCH38 and in sarek it's GATK/GRCh38, which is why in sarek we changed the key in
igenomes.config
to
GATK.GRCh38
p
I suppose if we're changing the underlying references then it's not such a bad thing to have a breaking change in the key 🤔
this 1
m
2. a/ Not a problem, I wouldn't propose a new structure that breaks all, we can keep params at the pipeline level b/ structure is flexible and planned to evolved My idea of having it flexible and versionned is that we point to a single yaml file for all references data/metadata for a particular genome, and whenever we update the corresponding schema or yaml or data, we update how we deal with it, but I do think we should deal with these references outside of the pipeline, so that we don't have to wait on a pipeline release to use a newer genome
j
But since #CELEMAGC9 was never updated, #CE8SSJV3N is not using it
Not quite true. We keep telling people not to, but since it's so gosh-darned convenient they're doing it anyway, people don't RTFM.
🤣 2
m
Best practices rnaseq is not using it