I’ve opened a PR to update the CADD module: https://github.com/nf-core/modules/pull/10290. The main challenge is the container size (~19GB) due to the required environments and models. This causes GitHub Actions to fail, even though the

docker

and

singularity

profiles pass locally. Any advice on how to handle these heavy-weight containers?

I don't know if the CADD package itself changed, but looking at the old module code it seems like the data wasn't in those old containers and had to be supplied via

annotation_dir

. The module should return to that pattern if at all possible, the whole reason being that 19GB containers and containers with packaged data are a big no no

CADD seems to have undergone a significant update between versions 1.6 and 1.7. The newer release includes additional annotation models and introduces ML framework dependencies, which have increased the overall footprint of the container image. The container does not include any data. We still need to provided those separately to the script.

Hmm, I can't say I know anything about this tool, but that seems very unideal. There is something similar with DeepVariant where the GPU container is huge (9-10GB) but fortunately the CPU one is only ~1-2 so that is used for the small CI tests. I wonder if it is possible to build a more minimal container for CADD. If it is and the default on BioConda is to just install everything that is kind of annoying

I totally agree. 9-10GB for a base image (which expands to ~19GB once pulled/unpacked) is definitely not ideal. I’ve hit a wall trying to slim it down further without breaking the tool's dependencies. I'll wait to see if @Simon Pearce or the someone from the core team has a clever trick to handle such large images.

That's crazy

The issue is that the models are expected to reside into the software environment itself for v1.7. I did try to gut the container, but it gets very messy with how the script expects its internal structure.

Hmm, we can't even easily stub test it if the image is that big. I guess we could override the container in the stub test

That sounds like a good compromise. Perhaps, I will use the old container for test. What do you guys think?

it's for stub, do whatever works easily and fast

Done! PR is now ready for review https://github.com/nf-core/modules/pull/10290

I don't like testing it with the old container tbh. doesn't really tell you anything about the functionality of the module...

I’m with you. The thing is, even if we manage to run the new container somehow, the required annotations are substantially larger, which makes it very difficult to test anything beyond stubs. So I think our only realistic options are either to increase the memory available to the CI tests so we can run the stub tests with the new container, or to run the stub tests with the old container.

I think I would then prefer a very basic container, e.g

container = "nf-core/ubuntu:22.04"

to make it clear that this is not using CADD at all.