https://nf-co.re logo
Join Slack
Powered by
Hi everyone, Can I get some eyes on this pipeline ...
# team-maintainers
f
Hi everyone, Can I get some eyes on this pipeline proposal please? github.com/nf-core/proposals/issues/176#… I talked to the developers in person at ASHG in Montreal and went through their proposal and it adds a novel flow for nanopore to nf-core really doesn’t clash with oncoanalyser much. Stephen one of the main developers of oncoanalyser agreed with this as well. So I would appreciate some feedback or ✅ for them to know how to proceed. Thank you keanu thanks
👀 1
m
looks good, we've been talking about what to do with the many longreads prosals
f
@Maxime Garcia what was the TLDR for tools like Dorado etc?
m
I'd say:
Basically all ont pipelines start with the output from a separate Dorado run
So start from bam format
f
I’ll leave them a note on the pipeline then. So we wouldn’t be supportive of a pipeline that has Dorado in it at all or can they just add notes etc to say, “ONLY USE DORADO FOR RESEARCH PURPOSES?”
I imagine this might have implications for their live analysis arm though
I appreciate this insight @Maxime Garcia
m
yeah, dorado is a mess
I'd say @Arthur and @Kevin Brockers might have the more insight here. From what I read, we might be agreeing on dorado, but I would be cautious personnaly
f
Yeah I wrote up some guidance on this for them on the proposal but happy to wait to post to hear back from @Arthur and @Kevin Brockers about what was actually the outcome of the license discussions
a
I would say that pipelines for ONT should be able to start from a uBAM/BAM even if they also end up supporting pod5 starts with dorado. It's quite common for Nanopore users to receive post basecalled outputs if they get their sequencing done by a provider. In terms of pod5 start via dorado I would say it should be added as optional route if the general consensus ends up being that we are ok with dorado being in nf-core (an ongoing discussion)
party this 2
f
Can there be a shared starting subworkflow for all these pipelines?
👍 1
a
If we approve having dorado in nf-core then I think with workflow chaining that it should be a standalone "dorado" pipeline probably
💯 1
👍 1
f
@Arthur thanks for the details. Why would we need a completely separate dorado pipeline if we acceept dorado inside nf-core? In this specific pipeline it looks like dorado is one module. If we are permissive, why to through the hassle of an entire separate pipeline rather than a module and leaving some note of saying “Dorado path only for research”?
a
I think in the direction we are moving - pipeline composition - it makes more sense than a subworkflow. It would probably be a sort of a generic ONT basecalling/QC pipeline imo. I think given the difficulties of ONT software isolating it to a single pipeline is also easier than having it seep into every pipeline that can feasibly ingest ONT
👍 1
m
pipeline compo for the win
f
I guess my problems with this are: • pipeline chaining not yet standard in nf-core so doesn’t help them in their proposal for now • why a full pipeline if in their pipeline its currently just 1 module (sorry I don’t know well enough what entails a “dorado” run)
So for the current proposal, I would like us to make a call on whether they can leave dorado in or need to take it out to be an nf-core pipeline. Also, I feel like none of this is a blocker for acceptance into nf-core. They are just looking for guidance and will likely follow whatever we suggest!
a
I would say take it out until we have consensus but that's just my 2c
👍 1
f
How about we decide on whether we accept the pipeline into nf-core in general. They port over their existing pipeline and before the first release we can still make that call to remove it? Feels like if it’s not clear for now, no need to rip it out if our decision on dorado might change?!
f
I am still unsure how this relates to all the other longread pipelines. There is an overview now I would rather double check that first before we port existing pipelines
f
From my checks, it doesn’t overlap much with other longread stuff in nf-core.
But yes, of course everyone do your own assessment
f
There is just a bunch of open proposals all around that topic (or at least that is my impression at the moment). We did something like this before when we had multiple at the same time I think @James Fellows Yates came up with a comparison table that needed to be filled. But if everyone else says there is no overlap we can go ahead 🙂
f
Ahh ok, let me do a check on other open proposals versus this for overlap!
👍 1
j
I did what???
f
I remember some table we did when we had multiple proposals all going towards a similar goal but maybe I am hallucinating 😄
🤷 1
f
image.png
👀 1
Had Claude give a look over all the pipeline proposals using long read sequencing. As you can see above, many share some core calling components and then oncoseq has a few differentiators, only adaptive sampling being shared with ont_dap. ont_dap. Ont_dap is a snakemake port with no Nextflow code yet and oncoseq is a fully written Nextflow pipeline already.
a
Lost my mind there... There's too much overlap on all these for my perspective. I think some or all of them should be combined. I would need to dig a bit deeper but wouldn't real-time (listed as a differentiator) be dependent on real-time basecalling ie dorado?
💯 1
f
That was basically my question about how real time is related to dorado above. I have never done ONT analysis so I don’t know, would have to ask them specifically again.
Coming back to overlap, oncoseq and ont_dap work from tumor samples alone, while the other two need matched normal samples, so I do think they serve very different purposes. Oncoseq’s main purposes seems to be fast / live tumor classification based on methylation data. Ont_dap adds some functionality that could be added to oncoseq. longtumourprofiler and LongCaller however use normal-tumour pairs and so their variant calling stack is different from the other two in my understanding. So for me it would make sense to suggest ont_dap to join oncoseq (under a new name) and the other two to join as well, but NOT to join all four into one!
These are the tool stacks the two (single-sample vs paired) use respectively and why they don’t overlap: • ClairS-TO for single samples versus ClairS/DeepSomatic for pairs. • Sniffles2 versus Severus/SAVANA. • Read-depth copy number with QDNAseq versus ASCAT/Wakhan, which need the normal to estimate purity and ploidy.
f
I mean one can combine somatic and tumor-only into one pipeline it has been done before
maxime 1
😆 1
s
Sarek is using it
f
My point wasn’t that it was impossible but that it’s gonna create one big monolith pipeline that will be hard to maintain and coordinate. Just my personal opinion though.
f
I am fine with both but having multiple per mode needs to be prevented
k
Sarek real time analysis is incoming? I think, depending how this has been implemented it could still be a nice distinction.
😆 1
f
I think suggesting to 4 different groups to all agree on a common workflow layout, which tools to include, how processing is handled, what samplesheets look like etc would be a logistic nightmare and ultimately slow down development for that one shared pipeline. Happy to be proven wrong, but I think merging the two that do tumor only and the two that do tumor-normal and encouraging both to make ONT tooling nf-core modules and subworkflows as much as possible is the better way to go.
a
But that's basically the tenet of nf-core
💯 1
One pipeline per data type
I don't really see tumour only vs tumour normal as being different but just subsets
f
Also are we distinguishing between ONT and PacBio?
f
What do you mean @Famke Bäuerle? All four of these are ONT!
a
It's kind of murky, they have a lot of overlap, but a lot of tools exist that only do one or the other of them
f
Yes I know but fundamentally both PacBio and ONT do longreads right? But I know we discussed this before but I do not remember what we agreed on. But I believe there are longread names pipelines already that plan to include PacBio and ONT but again I could be wrong
a
Yes there are some that support both
f
So I think we should go through this carefully and rather bring people to join forces on a few pipelines than having a bunch of pipelines trying to solve similar problems and then getting abandoned because they do not have a maintainers team
I am not an expert in long reads but I think we should check which pipelines do both (or plan to do both) and then compare also them with the current proposals to come to a good decision
I am not saying we should merge everything into a sarek-like longread pipeline but I am saying if we have essentially the same preprocessing for all then we do one longreadpreprocess then a tumor-only a somatic and a germline calling pipeline (can be three different ones) and one downstream pipeline (that would just be my very generalized view)
@Florian Wuennemann can your claude add these other longread pipelines to the comparison? 🙂
👍 1
f
Sure, I was just doing a potential metro map for combining all four proposals
👍 1
That’s a suggestion for combining all four porposals, now let me see which other long-read pipelines have overlapping functionality
💯 1
🤯 1
m
My take, we merge all long_reads in sarek it’s simple as that 🤣 (do not listen to me here, this is an obvious bad joke, we really do not need to add more to sarek at the moment, especially long reads)
maxime 1
s
Sarek is using it
f
I suspect the shared by all has subtle differences in parameters between the proposals, so maybe not as overlapping as this map suggests but hey, maybe they can all use this common flow
👍 1
a
I think the future looks like smaller modular pipelines chained together. Which is what Maxime plans for Sarek I think. That is easier if the overlapping parts are already aligned than if 3+ pipelines all implement subtly varying eg alignment, annotation etc
👍 2
this 1
Which is to say I think even if combining them ends up being a bit bloated in the short term, it might be politically easier if that future arises.
👍 1
f
That was also what I was trying to hint at!
💯 1
m
Which is what Maxime plans for Sarek I think
I’d love to claim it, but I think this plan was made so long ago, even I don’t remember who first had that idea. But latest nice plan with a nice design is fully Famke
sarek wine 1
j
I think the future looks like smaller modular pipelines chained together
I’ve used this as an argument on mulitple proposals over the last 6 months, so you have a lot of precedence if you need
(or hwoever you spell that word)
Open in Slack
PreviousNext
Powered by