I have a docker image that is 4.7GB, which apparently is too big for the singularity tests to run for my pipeline. What would be a reasonable size for a container? I'm trying to determine how hard I should try to reduce its size

too big in the sense of your CI is throwing a "no space left on device" error?

Exactly 😕

I think most would treat Docker containers with 100MB or less as ideal, 300-500MB decent for regular applications. If the container contains model weights for AI models, I guess Gigabyte ranges are normal. I had a quick glance at the Pixelator Dockerfiles and they look pretty solid to me with a lean

runtime-base

and just copying the build outputs over later. If this other image is build in a similar fashion, I guess there is not much you can optimise other than maybe quantise your params?

The container in question is actually from our R package (https://github.com/PixelgenTechnologies/pixelatorR/blob/main/Dockerfile), so no AI here (yet). It seems most of it comes from R dependencies. I've tried to throw an LLM at it, but so far without success... In any case, thanks for taking a look, it's much appreciated! 🙏

On the first glance, this Dockerfile is obviously a single-step one that keeps all build dependencies around. But I have to admit that I have only once packaged an R-application into a Docker container, so I do not know whether the

-dev

versions of the libraries are really needed post-install. Packages that are installed from CRAN or build are placed into a the library. Inside R, you can normally enter

.Library

to get that path. Maybe copying over that directory onto a fresh image derived from r-base suffices? Apart from that, I notice in the dependencies that the application uses five different formats for storing large tables: Arrow, DuckDB, HDF5, tibble and SeuratObjects. Is that only for import/export or does it do computations in all of them?