https://nf-co.re logo
Join Slack
Powered by
Hey folks - a couple of days ago I chatted in here...
# mag
w
Hey folks - a couple of days ago I chatted in here looking for a side project to get more involved with bioinformatics, and MaxBin2 was mentioned as a known pain point in the pipeline. Following the methodology of rewrites.bio, I've reimplemented it in Rust: https://github.com/werner291/maxbin-rs It produces identical output to the original on four test datasets (B. fragilis, minigut, CAPES_S7, CAMI I High) and is runnable via 
nix run
without manual dependency setup. An accompanying write-up on methodology: https://github.com/werner291/maxbin-rs/blob/main/PAPER.md If anyone has the time to take a look, I'd appreciate feedback - especially on whether this would be useful as a mag module.
d
Hi @Werner Kroneman! Thanks, this sounds exciting! I can do a large-scale test with multiple samples over the weekend or the start of the next week. For the meantime, could you publish the tool as a container in the github registry? That way would be easier to integrate into the pipeline.
w
Sure!
Do you mean just a Docker container?
d
Yep, in grch.io or dockerhub, whatever is fine
w
alright, one container, coming up
https://github.com/werner291/maxbin-rs/pkgs/container/maxbin-rs - here it is, gonna figure out how to integration-test it
🙌 1
I haven't really run it as a container before and the tool tends to write random files (copied that behavior from the original), so lemme see
Yeah I'm already getting crashes inside the container lol - hold off a sec before using this thing
d
Ok!
w
Replaced 0.1.1 container with a version that appears to work when run in a nix vm test
👍 1
Hm, I found some significant gaps in my test coverage - it's possible you'll find significant failures, so I'mma dig into them
Okay, dropped some very major balls, it seems, lol - gonna have to fix the implementation - don't bother testing this yet
d
Ok!
w
Do you have a relatively small dataset and flags that would trigger recursive binning in the original? I'm having trouble with long runs, and the small toy datasets I have don't produce enough runs to trigger recursion
I downsampled one of my sets - good enough to test on and debug
Okay, went down quite a rabbit hole... Floats are funny 🙂
I had a reproduction failure between Rust and C++ - turns out there was some probability threshold of almost exactly 0.5 down to rounding errors in the 16th decimal place, which is about exactly where the long double from C++ on x86_64 diverges from the f64 I'm using in Rust
Technically not exact emulation, but in a place where it's not scientifically very interesting - I'll document it somewhere
That behavior will diverge for the C++ version between operating system and CPU architecture too - that would never reproducible with the original either
Okay, so, I pushed out v0.1.2 - recursive binning now appears to work, needs testing on more data which I can still do on my laptop at the moment.
Assessment: needs work, also the docker CI test is failing. Software is fun, lol
https://github.com/werner291/maxbin-rs/releases/tag/v0.1.3 Okay, @Diego Alvarez Saravia if you wanna try it this version should actually bin recursively. I suggest trying it on some smaller dataset first to see if you get even remotely the right thing, as you'd be the first to run this thing outside my laptop
So, something that might be interesting to pipeline developers especially, is that maxbin2 itself is a pipeline, one calls out to other tools that then become obligatory dependencies - I'mma add a feature where you can call just the EM algorithm and nothing else, and supply its inputs directly from whatever tool you choose. (The existing dependencies were heavy)
The original pipeline is of course still there for anyone who wants it
Open in Slack
PreviousNext
Powered by