Hi all, I've got a question that came up during <@...
# modules
f
Hi all, I've got a question that came up during @Simon Pearce’s review for this new module PR: https://github.com/nf-core/modules/pull/8635 Guideline 7.5 (tools without Bioconda) states that "If the software is not available on Bioconda a
Dockerfile
MUST be provided within the module directory." What I have done is to include the following simple Dockerfile:
Copy code
FROM quay.io/nfcore/force:3.8.01
Is this enough for fulfilling the guideline?
j
But what was the Dockerfile used for uploading to quay.io :thjink
🤔
I think the purpose of the Dockerfile is to build that thing that was uploaded there
f
I could use the one used to build the original tool container (https://github.com/davidfrantz/force/blob/main/Dockerfile). Would that be the right choice?
j
Yes, maybe with a README saying we have permission from DAvid
Given his license is copy left
f
It's GPL 3 so I guess that would work. But I feel like this is just a more complicated way of using the Dockerfile that provided.
j
Maybe, it's more lets say quay.io dies
and we don't have our copy anymore
f
I hear you but the original FORCE Dockerfile also has a reference to another image hosted on dockerhub, which could also die/be taken down
a
I don't think this is a hard requirement. For eg DeepVariant there is no conda or even a singularity. There is no dockerfile and we simply use the Google supplied image
shakefist 1
j
Fiiineeee
(i.e. Arthur is right, not a hard requirement, I'm just being nitpicky 😬
f
I think it says MUST in the guideline 😅
a
Technically the dockerfile for that is on the deepvariant page but I don't know if it is easy to rebuild
j
(guess who wrote the guideline 😬 )
The
FROM quay
satsifies the requirement of having it 😉
a
I do think the question should be "can we reasonably get this as a simple dockerfile"
this 1
👍 1
s
So how did the current quay.io/nfcore/force:3.8.01 get there?
j
A core team member uploaded it
f
Original image is from dockerhub: https://hub.docker.com/r/davidfrantz/force (we have permission to reupload)
s
why are we not just using that image?
f
more guidelines: https://nf-co.re/docs/guidelines/pipelines/recommendations/custom_containers The rationale was that we want to ensure that the components using that container still work even if the original image becomes unavailable for any reason.
s
then the Dockerfile should presumably be the file as it was when that was copying was executed. Although I'm not sold that should be within the modules repository, potentially that wants to be somewhere entirely separate
f
Sure, we can use the original Dockerfile. Some other modules also have Dockerfiles in the modules repo.
The problem with using the original Dockerfile is that it must be executed within the folder of the original git repo (https://github.com/davidfrantz/force) as it requires the resources in there. I think that would really bloat the modules repo. I refer back to @Arthur's point: "can we reasonably get this as a simple dockerfile".
s
Yeah, we don't want to copy all the resources in. Then I don't think we gain anything from having a Dockerfile here, apart from tracking where the quay.io image actually came from
So what we really want to do is just make it clear that our quay.io image is based on that other image, we don't need to be able to rebuild it we just want to show where it came from
f
Not sure if I understand your point. Do you still want to put the original Dockerfile (the one requiring extra files) into modules? The module guidelines suggest that this will be used to create some container, so the Dockerfile in modules should be usable.
j
I'm concerned if David deleted the image then what?
a
I mean the same is true for eg DeepVariant above if Google did that? Ultimately I think this is a best-effort thing and if the image were deleted then we cross that bridge when we come to it in terms of rebuilding in the future
j
Fair
I dunno feel free to update the guidelines if you want
#small-mound-i-feel-strongly-about-but-not-enough-to-fight-everyone
a
But maybe depends how widely used the module is?
j
Well I think force is almost used the entire pipeline right @Felix Kummer?
f
Yes, but again, we have the quai.io copy. So David taking down the image will not really affect us.
This discussion has deviated a bit. My intention was to discuss @Simon Pearce's comments regarding the Dockerfile on this module PR . @Simon Pearce you're still on Changes requested on the PR so I assume you're not satisfied with the current state. I think fundamentally there are two questions: 1. Should there be the Dockerfile in the modules repo at all?
j
But that just pulls the David one... But like I said I won't fight. I don't see the harm of it personally
But f it bothers you theb
f
This discussion has gone off track a bit. My intention was to discuss @Simon Pearce's comments regarding the Dockerfile on this module PR . @Simon Pearce you're still on Changes requested on the PR so I assume you're not satisfied with the current state. I think fundamentally there are two questions: 1. Should there be a Dockerfile in the modules repo at all? 2. If so, is the Dockerfile referring only to the quai.io image sufficient? My pov is that the guidelines clearly require that file exists and the current solution is the only feasible solution to have a functioning Dockerfile. If this requires a broader discussion of the guideline then I'm not confident that this is the right place.
👍 1
j
Fair point - @Felix Kummeras your first #team-maintainers task, please copy this thread to the maintainers channel with a TL;DR to kick off discussion. In the meantime I vote for option 2, if Simon is swayed by democracy here :👍
f
I just went through some of the messages and there seems to a misconception either on my side or on @James Fellows Yates's. If we copied the image to quai.io, as described here, it is fully independent from the original image hosted on dockerhub, no? (i.e. David removing it has no effect)
💯 1
j
True, but I still want to have a source of how it was built
Lets say David stops updating, a security bug crops up in our existing quay.io one, we have no way of rebuilding it with a fix for example
f
I see, that makes sense
j
So e.g. for now saying FROM dockerhub/David/force would be sufficient (sorry I was mistaken above) to describe how we got our own container
✅ 2
f
ok i pushed that to the modules PR
Now trying to figure what the TL;DR is 😅
j
When/where should we require a dockerfile, and what goes in it
👆 1
✅ 1
s
In the meantime I vote for option 2, if Simon is swayed by democracy here
Not sure what you are implying here James 😉
j
nothing