This might not be exactly an nf-core question but ...
# nostupidquestions
j
This might not be exactly an nf-core question but it's somewhat related. I noticed that modules using Wave containers (community.wave.seqera.io) seems to fail when using Singularity with
ext.singularity_pull_docker_container = true
It looks like the
PATH
environment variable is supposed to be updated by
shell-hook.sh
which is called in the
ENTRYPOINT
section in the Dockerfile, but
singularity exec
doesn't invoke the
ENTRYPOINT
section which is translated to
%runscript
when converting from OCI image to SIF. The native Singularity images don't have this issue because the
shell-hook.sh
script is invoked in the
%environment
section. While this is not a critical issue, it would be nice to be consistent with where and when we update the environment variables and to allow users to run OCI-converted Wave containers with Singularity.
Is this perhaps worthy of an issue under
seqeralabs/wave
?
p
If you could please open an issue there with a minimal repo / more details that would be great, yes please 👍🏻
This is just when using Singularity with the docker images? Not native singularity builds from Wave?
I guess very few people do that now, as in nf-core we usually generate both container types for every module. So folks run on the native images with Singularity 99% of the time
j
Makes sense and yes it only happens when using Singularity with Docker images (and specifically Docker images where the environment variables are updated with the ENTRYPOINT instruction instead of ENV)
👍🏻 1
s
@Junxuan Gao, can I ask why you are using
singularity_pull_docker_container
? Because I have been suggesting to remove this from the template, as I didn't see a use-case for it.
🔥 1
(and nobody has flagged this in the last ~12 months or so of having Seqera Containers in many modules)
j
@Simon Pearce When we deploy our pipeline, we pre-cache all the images used by the pipeline into a shared cache directory to avoid the per-user blob cache that is used by singularity by default. We then point the Singularity cache dir for the pipeline to the shared cache. We use the Docker images instead of the native Singularity images on Wave because the native Singularity images don't have unique names (just
data
if you try to run
singularity pull
with
--disable-cache
)
p
The nf-core CLI has
nf-core pipeline download
for exactly this task
It fetches the singularity images for you and renames them to avoid the clash
You probably want
--compress none
and
--container-cache-utilisation amend
Native Singularity images are typically more stable and can even be a little faster* (citation needed), so we generally recommend using them over Docker->Singularity if possible.
j
Thanks I will take a look
s
Please let us know if that is suitable for you, because I'm still keen to remove that
singularity_pull_docker_container
option if possible
j
That being said I think the issue I brought up regarding Wave using different directives for Docker vs Singularity is still worth looking into.
p
Yup, good to fix that either way - thank you 🙏🏻
s
@Junxuan Gao, did you fix your issue? I'm still keen to remove
singularity_pull_docker_container
in a future update