Hi @Phil Ewels @Sateesh_Peri @Felix Krueger We'd like to contribute two upgrades to the TAPS path in methylseq (

--taps

/

--aligner bwamem

). Before we open PRs, we'd like to agree on the approach. The one thing that needs your input is the rastair2 license (see below). 1. bwa-mem3 in TAPS mode as the default TAPS aligner • We'd replace BWA-MEM in

FASTQ_ALIGN_DEDUP_BWAMEM

with bwa-mem3 running in its TAPS-aware mode (

--meth=taps

). • bwa-mem3 is on bioconda (v0.14.1), so the container and conda setup is standard. • We've been running it internally on TAPS data with good results. • Things to be aware of: ◦ It needs its own index (

bwa-mem3 index --meth

), so existing pre-built

--bwamem_index

directories won't work with it. The pipeline can build the new index from

--fasta

as it does today. ◦ It uses more memory than BWA-MEM: about 80 GB plus about 1.2 GB per thread on hg38 in TAPS mode in our runs. • Question: are you OK with this becoming the default? Should we keep BWA-MEM selectable for users with existing indexes or tighter memory limits? _2. rastair2 as an alternative caller in `BAM_TAPS_CONVERSION`_ • rastair 0.8.x stays the default. A flag (e.g.

--taps_caller rastair2

) switches to rastair2 instead, and the two never run together. • rastair2 calls methylation, SNVs, and optionally indels together and writes a VCF. • The new modules would go to nf-core/modules first, then be installed into methylseq. • Parameters we'd expose (names open to discussion): ◦ SNV calling: always on with rastair2 ◦ `--rastair_indels`: experimental indel calling, opt-in ◦ `--rastair_single_end`: opt-in ◦ Strand guessing: opt-in. Could this reuse the existing

--non_directional

flag instead of a new parameter? ◦ `--rastair_skip_ml`: the ML model is on by default, and this turns it off Licensing How rastair2 is licensed and distributed: @Benjamin Schuster-Boeckler • It is free for non-commercial use, but commercial use requires a license from the University of Oxford. • There is no license key or runtime check, so it runs like any other tool. • A public image is already on Docker Hub. • Ben (the author) has spoken with the bioconda folks, who were OK with it being published there. That would give it a standard biocontainers / Seqera community container. What we'd add to the pipeline documentation: • A clear note in the methylseq docs, and in the module

meta.yml

, saying that commercial users must contact Oxford for a license. Questions 1. Is a tool with these terms acceptable in nf-core/modules and methylseq, given it's opt-in and the default TAPS path stays fully open source? 2. Is a docs note enough? Or would you also want a warning in the run log when

--taps_caller rastair2

is set? 3. Should we wait for the bioconda package before submitting the nf-core/modules PR, or is the Docker Hub image acceptable in the meantime? 4. Would you prefer the bwa-mem3 change and the rastair2 change as separate PRs? We're happy to jump on a call if that's easier. Thanks!

Rastair2 is now essentially on par with DRAGEN in terms of speed and accuracy for indel and SNV calling, and is similarly fast even without custom hardware

Quick thoughts: • Not sure off the top of my head, but do we have bwa-mem methylation-aware indices on AWS-iGenomes? I know we have Bismark there. If the pipeline currently always builds and doesn't have static references then no issue. If it can pull from iGenomes then it'd be good to know. ◦ Are you suggesting bwa-mem3 for just the taps workflow? Can we use it for bwa-meth bisulfite workflow as well? ◦ Have you looked into any other recent bwa mem rewrites, out of curiousity? mem3 is the fulcrum rewrite, right? • I'd rather not have both rastair versions, I think? Better to just upgrade to v2 and if people want the previous version they just use an old version of the pipeline? Or does rastair2 have different licensing? In which case maybe worth keeping both. • Licensing issues are honestly a major pain and I would much prefer to avoid non-OSS tools where possible (in my experience it rarely works out well, see GATK and others). But ok... There is precedent in nf-core for having a note in the readme, and having docker images available for CI etc. means it should be technically fine. ◦ Because nf-core is OSS and all other software in the pipeline is OSS, my main concern is that the large number of industry users we have might use it by accident. It could be good to have an explicit opt-in parameter for this reason?

--rastair-licence-compliant

or something? ◦ Side note @Benjamin Schuster-Boeckler as you're on the thread: it'd be good if the license was much more obvious in the docs. I see a note in the homepage but only when I went looking for it. It'd be nice if it were more obvious and maybe had a dedicated page? ◦ Related: it'd be nice if it were a standard license type, and if some of the terms were less broad (what does "commercial" cover? according to Claude "It covers any research aimed at a product, software or not, so pharma and biotech R&D is out. It also covers paid services to external organisations, which catches core facilities and CROs that charge for analysis."). But I assume this is probably out of your hands.. ◦ Homepage has two broken links with what looks like a placeholder for the repo name in the license info. There are other placeholders too, eg. mastodon So to your questions: 1. Yes it's acceptable 2. Warning in the log please, and maybe an explicit opt-in flag as above 3. Bioconda, but it's already there ✅ 4. Yes, separate PRs please.

I guess rastair is OSS, but not *F*OSS 😅 I understand this is a bit of a tricky bit, but there was a sense that there should be some return to support the ongoing development or rastair 🙈

Maybe OSS but not by the OSI definition.. (which is the only one that really matters). Yeah I've seen this happen a bunch of times, intentions are almost always good but in my experience it: (a) recovers next to no money and (b) causes a headache for other OSS maintainers and (c) makes people avoid your tool 😞 The reality is that it's almost never worth the legal fees for a company to negotiate with the university, so almost all just.. don't.

we're just going through that experiment 😄

If you're not familiar with the whole GATK story then I recommend reading this, it's a nice writeup from a long time ago, bitching about Broad + GATK (who finally went full OSS). It pretty much describes the situation you're in I think and highlights some of the probably-unintended consequences that this approach can have.

I remember talking to Geraldine on the GATK forums 10 years ago 😅

True but companies are very scared of these kinds of licenses. It's the number 1 blocker for adoption of OSS tooling (getting better over time, but still). Investors and auditors are super hot on checking this kind of thing.

Regarding bwa-mem3: it's the only bwa-mem rewrite that I'm aware of that adds a CpG-specific ambiguity option

Nice 👌🏻

Thanks for all the discussion! I will make the PR to nf-core/modules for 1) bwamem3 and 2) rastair2 (we can later switch the image and add the needed documentation about licensing as desired). Once these are approved I will make the PR to methylseq/dev and we can see from there? Please keep me in the loop for any wants and needs.

Sounds great!

Ping @Nils Homer, we've been chatting about TAPS for a while. I can also help drive the bwa-mem3 effort, we're using it on an internal TAPS workflow

@Michael Heuer interesting, thanks for sharing. Having had a chat with the tech-transfer team here yesterday, and pointing out these potential barriers to adoption, they basically suggest a simple flat-rate model, ie "pay X if you use this more than 1000 times a year", kind of like what you'd get with any SAAS, thus foregoing any complicated negotiations and providing a pretty simple calculation (ie "using this adds at most X per sample, but saves us Y because it's faster/we don't have to re-invent the wheel". Sounded reasonable to me. The only real alternative, ie DRAGEN, is a lot more expensive, so it seems fair to ask for a bit of a contribution to development and maintenance

Hi @Michael Heuer I just updated bwamem3 in nf-core/modules to be able to be run with taps data usijng the --meth=taps. I am about the add rastair2 there too, and once both of this pass I will make a PR to methylseq/dev to implement all these changes.

> I just updated bwamem3 in nf-core/modules Yep, saw that, nice work! It doesn't appear to pick up the .0123 seed reference

[bwa-mem3:--meth] original reference bns/pac loaded for extension (1 contig(s)).
[bwa-mem3:--meth] pac-fetch: original reference unpacked from .pac on demand (.0123 not loaded).
* [--meth] seed reference `.0123` not loaded (extension uses the original reference)

I'm also getting this on docker

$ cat .command.err
[W::hts_set_opt] Cannot change block size for this format
samtools sort: failed to read header from "-"

Investigating 👀

Thanks Michael! 1. bwa-mem3’s own

index

help says this is by design: 0.16.0 no longer writes

.0123

unless you pass

--emit-unpacked-ref

, and

mem

reads bases from

.pac

as it needs them, so it “never reads”

.0123

. As a check, I built a plain index into the same folder and realigned, and the SAM output was byte-identical. 2. Those two samtools lines mean bwa-mem3 exited before writing a header. I get exactly that

.command.err

when bwa-mem3 is OOM-killed (exit 137). Could you share

.exitcode

, the full

.command.err

and

.command.log

, the reference, and Docker’s memory limit? A human

--meth

index needs roughly 30 GB at alignment time.