Can someone explain to me what they expect the ont...
# team-maintainers
s
Can someone explain to me what they expect the ontologies field in modules meta.yml to actually contain.
j
a list of ontology links https://nf-co.re/blog/2025/modules-ontology (I can't link to the exact example but it's the very last code snipet if you scrolldown)
s
Yes, but how do you expect people to do anything with that?
The sam/bam/cram one for instance is just a list of meaningless numbers
how do you expect people to find the appropriate context for their tool they are adding
Take this PR for instance Now @Usman has put (or maybe the nf-core tool has automatically put) a fasta ontology for the input. But the other things?
j
with the new version of tool (now on dev) this is automated from the pattern, it reads the extension file and tries to find the ontology
s
Is there a plan to lint for this?
j
and it also adds the comment, so it's not only a link difficult to know what it exactly is
👍 1
and yes there is linting 😄 also the
--fix
option will try to add missing ones
you you can also do is go to the EDAM browser and look for the one specific for your file
there will also be documentation about this 🙏🏻 but because it's not released yet the documentation is not in the website yet either
👍 1
s
Basically what do I do with the current open PRs
Is my actual question
j
I think this PR is ok 👍🏻 you could also add an ontology for the yaml file https://edamontology.github.io/edam-browser/#format_3750 and suggest the author to check if there are other possible ontologies to be added, but I don't expect the reviewer to look for them 🙊
u
This was all automated, I think.
s
and suggest the author to check if there are other possible ontologies to be added, but I don't expect the reviewer to look for them
Yes, I don't have time to go through and sort through the ontologies.
The yaml file ontology should be in the template presumably?
j
yes, if I am not wrong it will be with the next release
I think the fasta one also comes from the template
p
Is the motivation for adding these clear @Simon Pearce? eg. what we intend to do with them once added? I'm not sure if we've communicated that very well yet..
s
No, I don't have any motivation frankly. I see them as something that has been decided will be a good idea and super useful and will turn out to never be used by anybody
p
Yeah that was what I feared you might think 😅
🤣 1
s
And in the meantime they will require time and energy to implement
I've seen these kind of databases and never seen them actually amount to anything because nobody uses them sufficiently
p
Maybe we should update that blog post to move the "future potential" to the top and bulk it out way more to better explain the motivation
But my take quickly:
We (I) mostly don't actually care that much about the EDAM ontology itself. I agree that wherever I've seen it used before it's felt very academic and not really all that useful in a practical sense. However, what I do care about is having some kind of identifier for different types of process inputs / outputs. Rather than inventing our own, it seemed prudent to use an existing standard.
Once we have these identifiers, we can start to build a whole lot of tooling. For example, automatically matching and validating which processes can go together. Use in the language server to validate channel types when you're writing code to identify mistakes in real time.
They will also cascade through to pipeline level, where they can be used in the workflow input and output declarations, to give better formal specs of what pipelines ingest and emit - this will be a crucial component for daisy-chaining pipelines (see #wg-meta-pipelines for more discussion).
Finally, almost as an afterthought, they can be used for better automatic documentation and integration with other ecosystems.
The Nextflow language itself is moving towards a stricter syntax, which will hopefully have a formal typing system at some point. I can imagine that transition using the existing ontologies that have been added to create custom Nextflow types for different channel file formats, for example.
Then we can start to be really clever and build a lot of nice developer tooling and automation around that with minimal effort.
But, the foundation for all this is having some kind of ontology. The ability to differentiate between a CSV, BED and Fasta input file. Currently they are all just a file input as far as Nextflow and the ecosystem are concerned. By adding ontological information we differentiate them.
We could wait until all that stuff is built, but then we just have a huge backlog of thousands of modules to add ontologies to sadpanda So I figure it's better to write automation to start grabbing these things automatically, which is what Julia has done (hopefully for the main part it should work in such a way that the developer and reviewer have to do very little as the relevant ontologies will be found completely automatically).
TLDR; Nextflow typing and pipeline chaining is coming. I'm pretty confident that ontologies will play a big part in how successful they are.
@Simon Pearce make sense? Or still smells of 💩 ?
🤣 1
s
Thank you Phil, that was very helpful
Maybe you should add basically that entire statement to the blog post
💯 2
p
Any questions / anything lacking / anything you disagree with before I do that?
s
I'm skeptical that the tools will ever actually get built that make real use out of it, but maybe I'll be proven wrong there. If it is as simple as just automatically guessing based on the file extension, then why do we need the ontology in the first place.
j
guessing from the file extension is a first approximation. The initial idea was to have a CLI helper tool (with nf-core modules create) that will let you select from a drop down and make suggestions. But this is a bigger project and we started by the simple automation
p
My original idea for this was that nearly everything would come from the bio.tools identifiers, which we already have in place from the bioconda package names..
s
But that require everything to have already been put into bio.tools
p
True
My gut feeling was that many / most things are, but maybe I'm wrong
s
I'd never heard of it before it was introduced through the tools
p
ah right, I've been wanting to make more use of bio.tools since 2019
👍 1
u
nf-core -v modules lint --fix samtools/fixmate
works with
pattern: "*.{sam}"
but not with
pattern: "*.sam"
p
Could you please make an issue on #tools about that @Usman ?
keanu thanks 1