Can I ask something generic for other <#C04QR0T3G3...
# pipeline-maintainers
y
Can I ask something generic for other #C04QR0T3G3H. We have a full branch of our pipeline that is not supported in Conda only Singularity and Docker, some users in the pipeline who previously used Conda for the other branches, with the new Conda license of research institutions are moving to Singularity. Releasing the conda package creates an extra hassle which could be mitigated by releasing Singularity on top of Docker. My question is, are any other pipelines wondering and considering not to support Conda.
👀 1
e
The
spatialvi
pipeline does not support Conda for Space Ranger pre-processing, as that software is simply not available in Conda and would be difficult to add there, so containers are enforced if the user wants to run those steps.
f
#C068T3R7LAE has never supported Conda, but we are a bit of a special case because our (non-bioinformatics) tools are not on Bioconda.
🤯 1
👍 1
y
This is also my case for DIA analysis. But for all the other branches of the pipeline that are supported using Conda, Im wondering if is really needed.
m
We also have a number of (genomics) pipelines that don’t support Conda because some software are not available in Conda. Some missing software could probably be added to bioconda, but others just can’t (only distributed as Docker images)
y
@Erik Fasterius @Matthieu Muffato and have you seen any decrease in usage because the pipeline do not support Conda, or users complaining?
m
yeah, unfortunately as much as we'd love to support conda everywhere, it's sometimes harder than that
👍 1
@James Fellows Yates correct me if I'm wrong, but it's more a STRONG recommendation than an absolute MUST?
y
I totally agree, we are also making the hard decision of moving away from it because maintaining it is kind of hard for #C02Q3FL29PD. However, I would like to know the experience of others.
j
Docker mandatory Conda is recommendation (but with fuzzy language to sort of encourage it)
y
@James Fellows Yates from nf-core do you have stats of how many pipelines are still supporting Conda? Out of curiosity.
j
I don't have stats. The only times I've heard of people dropping conda support entirely is #C068T3R7LAE All other cases is just certian modules/tools within a pipeline that isn't supported, not the entire pipeline
So to my knowledge the vast majority of pipelines do support conda
👍 1
y
Thanks for your feedback.
👍 1
s
The license issues for conda are only on the
defaults
channel as far as I know, which has been removed from nf-core pipelines.
this 1
👍 1
y
Yes, true, but looks (from the discussions I have with some users) that systems and getting panic in research institutions and removing the entire support just to do not get into legal issues with Conda
j
All the institutions in associated with just blocked the anacdona's default channel API
m
@Yasset Perez Riverol. No one has complained about the lack of support for Conda. 3rd-party usage is actually generally increasing because we’re promoting the pipelines, etc. We do get issues being raised for problems running Singularity on some HPC, but Conda has git its lot of problems as well anyway
🙏 1
j
They've not blocked conda entirely
m
(yep, exactly what our IT department did for Anaconda)
e
Nobody has complained on our end either
👍 1
p
I doubt people will complain if it's absent from the start because people just won't choose to use it at the beginning. Removing it later might cause more pain for users.. But I agree that it's a lot less important now than it was when we wrote that recommendation 7 years ago.
That said, I think that (bio)conda remains a good method for distribution of software, including containers. When combined with wave, we will soon be getting all of the following: • conda arch64 • Conda arm64 • Conda-lock (both) • Docker (both) • Singularity native builds (both) So basically 8 different platforms from a single conda recipe, automatically
We also have automated version bumping that works with conda, which again will be built out more - hopefully to from a conda version bump right through to modules / pipelines automatically
And finally, and perhaps most importantly for me, conda democratised software packaging. It's difficult / impossible for others to update docker images that you made. But conda recipes can take PRs from anyone.
y
Hi, @Phil Ewels thanks for your feedback. I totally agree and Im a maintainer of Bioconda. But we have some cases in which we want to speed up nf-core releases without waiting for the bioconda package release. We probably will go for a dual model where some releases of the workflow will have Conda support and others will not. The problem that we have, is that we have some dependencies that take a while for them to produce the formal release and Bioconda. Then, sometimes we want to do a release based on our own version of the tool (deployed in our container registry in GitHub). If we don't do this, what will happen is that the majority of the data releases with quantms will be using the dev branch (which is the one using this approach now). Im just explaining our use case for others that may be helpful.
👍🏻 1