thread1: gzip

I ran into this issue with a singularity container in

nf-core

modules recently and I am wondering if there is any plan to audit this and attempt to update the containers affected? The module in question avoided being flagged by CI because the tool only uses

gzip

when run under certain conditions.

I think this will come down to the base images used 🤔 to be on the safe side, I’m thinking a single source of truth should be used, which is then converted to the different formats for docker/singularity, without relying on separate base images

Yeah I commented questioning how this happened on the issue. It seems like a Seqera containers issue but from what I could trace Paolo basically just said “define all dependencies”

yeah, that seems to be the gist of it. I do think some kind of “stable base” should be available to be used across platforms

I can’t really fathom a situation where you’d ever want different base images.

But thats the current situation AFAIK

I know a lot of people do that already (and we have/had an option before for singularity to pull from docker in nf-core) but it is a blocker for people running

download

right? (or they at least then need singularity installed where they run

download

)

The thing is, I’d like to avoid cluttering all the env.yaml files with

gzip

only to satisfy the linter

This isn’t a linter thing though right? My main issue with the current “fix” is the the containers will only rebuild in the correct way if you know the custom base packages option or use

nf-core/tools

to build them, and right now it doesn’t work to rebuild images for a patched module installed in a pipeline. Would be better if there was a base image that we could point people too I think - but then that’s another thing we have to maintain and upgrade, which with the rapid pace of 0-days and stuff like this certs chaos is going to be far to often 😂

This isn’t a linter thing though right?

I think for most it is though. The linter requires

echo '' | gzip > file.ext.gz

in the stub, so if singularity doesn’t have gzip, they add it to the env.

But it’s not the linter that requires that, it’s that the stub tests will fail if you don’t

gzip

like that.

but then that’s another thing we have to maintain and upgrade,

There’s automation for that?

Sure, but that’s separate (and less bad) than the fact the whole test fails if you don’t gzip the stub data in some way

Huh? I’ve been using it for a while now, with success 🤔

If by success you mean singularity images have

gzip

then yeah that’s true, but it’s because the

pixi

base image now has it (

ubuntu:24.04

) and not because of

basePackages

.

But… how come singularity failed pre fix and worked post fix 🤔 there must be some other variable I overlooked then

image.png

Yeah… that

I only noticed it wasn’t in the installed conda packages because I tried to manually compare the lockfiles and then I couldn’t see

gzip

. I don’t really understand what happened either though, because we were using the pixi V1 template already…

oh, makes sense, come to think of it 🤔