https://nf-co.re logo
Join Slack
Powered by
Does anyone know the policy around making a new pa...
# bioconda
a
Does anyone know the policy around making a new package for an existing one? the GATK4 package is over 2GB 😱 but the actual main usable for stuff for variant calling is under 500MB, I was considering GATK4-lite but I don't know what the policy is around opening another package recipe in bioconda...does anyone know of any precedent here?
m
wait for the rust rewrite from fulcrum
🤣 1
m
doesn’t 
gatk-lite
already exist? I know 
picard-lite
does. The main issue is the 
lite
style packages don’t include external dependencies needed for several tools to work
also, keep in mind the gatk4 recipe changed names recently to 
gatk4-main
Also, I agree with Maxime. give it some time and you’ll be able to replace your entire workflow with piped fulcrum tools
a
doesn’t 
gatk-lite
already exist?
Surprisingly, no! github.com/bioconda/bioconda-recipes/pull/69697
Also, I agree with Maxime. give it some time and you’ll be able to replace your entire workflow with piped fulcrum tools
I agree, but that's not how legacy software works sadly.
The main issue is the 
lite
style packages don’t include external dependencies needed for several tools to work
Do you know which tools rely on R, Python and gcnvkernels? The pacakge is bloated so any way we can trim it down is helpful
m
I think most tools that output plots use an R backend for that
also, what were they smoking to ever think that was a good idea facepalm
Why is the 
lite
package not also 
noarch
?
a
looking at that now
I was very aggressive and bundled a JRE
which made it nice and smol 🤏 but was probably unecessary
Ideally, I want it under 512MB because that's the cache size of CDNs like cloudflare
😆 1
OK we can switch from jlink to openjdk which costs 200MB but is an easier sell as a bioconda recipe
hmmm wait - gatk4-main doesn't include the cnvkernels
m
say what now?
m
yeah, don't we add that most time?
m
I suppose I didn’t notice because we use wave to create the image for #CJRH30T6V
and the 
gcnvkernel
is defined in 
env.yml
a
<http://community.wave.seqera.io/library/gatk4-main:4.6.2.0--97d31e13e69d824d|community.wave.seqera.io/library/gatk4-main:4.6.2.0--97d31e13e69d824d>
Gaaahhh, it slipped in! No need for gcnvkernels at all right? except the 3 modules that use it
m
it’s only used for some tools AFAIK
m
Give the 3 of us the broad and we rewrite GATK the way it should be done
a
Pretty sure that's Sentieon!
m
too complex to run
I want an open source solution
m
@claude
fix dumb gatk design decisions
m
I mean, I could do without a license server
I want consistency in args
let's fully remove picard
m
lol 😄. we already have 
riker
for that
m
but picard is in gatk already, we do not need both
m
ah, like that. yes I agree
a
Right so gatk4-base is already about as slim as you can get before doing mad stuff, can we swap over to this image?
1 PR with 64 modules or 64 PRs with 1 module? 😅
m
Mega PR!
elmofire 2
a
One horse sized duck, got it
big red rubber duck 1
m
MEGA PR
a
1. doing the mega PR right now 2. keep exploring jlink and a gatk4-lite package to make it smaller in the future
There's an _un_markduplicates???
🤣 1
OK I've added a bundled gatk4-lite which uses jlink: github.com/bioconda/bioconda-recipes/pull/69697 it works, but I can't confirm it will work in practice, there may be too many random miscellaneous tools that are required. In the ☝️ modules PR I found loads of dependencies which were pulled in from gcnvkernel but not defined.
So for now, I'll try and get that modules PR merged, then we'll test with the gatk4-lite package when or if it gets released
m
There's an unmarkduplicates???
Yeah, I was surprised too when I stumbled onto it
a
OK here's the gatk4-lite package: github.com/bioconda/bioconda-recipes/pull/69697
and the phase 1 nf-core/modules PR is here: https://nfcore.slack.com/archives/CQY2U5QU9/p1790703348083889
@Matthias De Smet I'm confused by your gzip comment, nf-core tools doesn't add gzip as a base package (yet), so how will the image include gzip? Or worse, if we trust the base image we may get a random gzip package based on the Linux distro.
m
you can change the base image in wave
we've been doing that in other modules
a
So you can't just use Seqera Containers? sadpanda
m
The main issue (that the PR I linked fixes) is that 
gzip
is included in the docker base image, but not in the singularity one. I meant to pin it to one version for both variants with that PR
a
Is it worth keeping until Singularity is fixed? It's idempotent after all
m
Other than that, I’d just use #CE5LG7WMB to create the containers and all the other config files
m
can't tools dev switch this base image for us?
m
no idea, I just edited my local tools install with my fix 😄
I asked Matthias to take another look at the PR
m
Matthias is back today, so we can pester him again 😆
a
So will the Singularity tests pass without gzip specified?
m
if you apply my fix they will 🤣
a
😒
m
let’s summon @mashehu to motivate a merge
a
ngl I really don't want to have to deal with 4+ repos to get a slightly smaller container
thistbh 1
setting the robot off doing it.
m
> I really don’t want to have to deal with 4+ repos to get a slightly smaller container Ah yes, such is the fate of the bulk updater
all ye who enter
a
OK gatk4-lite test going: github.com/nf-core/modules/pull/13062
👀 1
✅ 1
m
looking good, but CI is failing
a
No gzip again, but it's good it works!
Updating container image
m
FYI, the #CE5LG7WMB fix was just merged, so if you use the latest dev version the gzip stuff should be fixed
🙏 1
excited dog 1
m
looking forward to this new container for gatk4
a
extreme teamwork 2
m
If you want to update sarek, now is the time, we are getting ready-ish to release
a
panic
2 Views
Open in Slack
PreviousNext
Powered by