What is the latest best-practice around containers...
# modules
a
What is the latest best-practice around containers for modules? I was creating a module yesterday and it couldn’t find the latest biocontainers singularity image. When I asked on Biocontainers it transpires they exist but that
the website is largely unreliable these days
🫠 1
m
I'd go for seqera containers then
a
What’s the process for getting those built?
m
Specify what you need. The architecture if needed
Docker or singularity and trigger the build
You get a uri directly for docker
And you need to wait to get the direct download link for singularity
a
What’s the policy here for use in
nf-core
though? Given support for
conda
is typically required?
m
I gave up on creating mulled containers on biocontainers since that happened
a
Only use
conda
available packages for
nf-core
modules?
m
You can still copy the conda env file and use it
Pip works too
But not supported by tools linting for now
The way I see it. It solves most of my issues with GATK4 containers
a
Will seqera ever do a docker (rate limit) or an anaconda (change license then sue)? asked another way, is there any commitment from seqera to keep access to their containers free (at least for public health non profits?)
a
Seqera commits to retaining images uploaded to the Wave Community registry at least for 5 years since their creation.
Seqera Containers is powered by Wave, so the Wave rate limits apply:
• Anonymous access: 250 container pulls per hour
• Access with a Platform access token: 2,000 container pulls per minute
party this 2
👍 1
a
That is very generous rate limit for anonymous!
j
The convert-bot from quay to galaxy was broken. But now it’s fixed again https://nfcore.slack.com/archives/CM46YC6BZ/p1746990250026229
You should now have your latest image available again
a
It was already available. It was built over a month ago. But it seems that the BioContainers website/API is unreliable and it doesn’t show up when browsing the registry
p
Their API breaks our CI and tooling a lot. I measured its downtime at one point and it was something like 20% this is fine fire
🤮 2
I really really really really want to finish the Seqera Containers implementation work that we've been promising everyone for months now. That will signify the end of our dependency on BioContainers and all of these threads should just magically go away, because it'll all just be automated and wonderful magic rainbow puke 🦄 🌈 📦 docker
💯 1
😆 1
• Step 1: Write a conda
environment.yml
• Step 2: There is no step 2.
😆 1
shared a GIF
j
Soon ™️
m
We need support for lock files. That doesn't seem possible at the moment
p
No, lock files will be supported
Wave already does them, we added support a few months ago
They're included in the nf-core roadmap blog articles too, idea is to take the lock files from the Wave builds and save them, then expose them to pipeline users under dedicated config profiles
m
But you can't use them with the
conda
directive as of this time right?
p
I... thought you could.. 🤔
m
I haven't been able to get it work
I took a lock file from a seqera container, and tried to supply it to the directive, and it just made errors
> To use with Nextflow, simply set the conda directive to the lock file path.
m
I've seen it, tried it, and failed
p
Did you make a GitHub issue? 😅
I wasn't aware of any problems
We can look into it though, for sure. It would be good to ship with that if at all possible.
m
p
I'm AFK at the moment (power cut at home) so can't test
a
What were the actual errors you saw though?
Were they NF errors or Conda errors
p
Summoning @Jonathan Manning to this thread as our in-house conda lock file champion
m
That took a while to reproduce ...:
Copy code
[-        ] TASK -
Creating env using conda: environment.lock [cache /Users/mahpa906/Sandbox/Nextflow_sandbox/work/conda/env-f858e1ce70c0395f4944a5e853266bfe]
ERROR ~ Error executing process > 'TASK'

Caused by:
  Failed to create Conda environment
    command: conda create --mkdir --yes --quiet --prefix /Users/mahpa906/Sandbox/Nextflow_sandbox/work/conda/env-f858e1ce70c0395f4944a5e853266bfe environment.lock
    status : 1
    message:
      Retrieving notices: ...working... done
      Channels:
       - conda-forge
       - bioconda
      Platform: osx-arm64
      Collecting package metadata (repodata.json): ...working... done
      Solving environment: ...working... failed
      /Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/gateways/logging.py:66: FutureWarning: functools.partial will be a method descriptor in future Python versions; wrap it in staticmethod() if you want to preserve the old behavior
        record.msg = self.TOKEN_REPLACE(record.msg)
      
      PackagesNotFoundError: The following packages are not available from current channels:
      
        - environment.lock
      
      Current channels:
      
        - https://conda.anaconda.org/conda-forge
        - https://conda.anaconda.org/bioconda
      
      To search for alternate channels that may provide the conda package you're
      looking for, navigate to
      
          https://anaconda.org
      
      and use the search bar at the top of the page.



 -- Check '.nextflow.log' file for details
using
pixi global install conda==25.1.1
Copy code
process {
    withName: 'TASK' {
        conda = 'environment.lock'
    }
}
conda.enabled = true
a
Going to guess the issue is that nextflow thinks that is not a path
Is the solution not simply
./environment.lock
or some other more-obviously-path string
m
Copy code
N E X T F L O W   ~  version 25.04.0

Launching `main.nf` [mighty_celsius] DSL2 - revision: 221a774568

ERROR ~ Error executing process > 'TASK'

Caused by:
  Conda prefix path does not exist or is not a directory: ./environment.lock



 -- Check '.nextflow.log' file for details
and same with the latest nextflow version
a
Ok, so what format is the lock file? try changing your lock file name to
environment.lock.yml/txt
m
Copy code
Creating env using conda: ./environment.lock.yml [cache /Users/mahpa906/Sandbox/Nextflow_sandbox/work/conda/env-b29a87e26dc74e81-fec495b0e47379d85cd966c886d2a4d3]
ERROR ~ Error executing process > 'TASK'

Caused by:
  Failed to create Conda environment
    command: conda env create --prefix /Users/mahpa906/Sandbox/Nextflow_sandbox/work/conda/env-b29a87e26dc74e81-fec495b0e47379d85cd966c886d2a4d3 --file /Users/mahpa906/Sandbox/Nextflow_sandbox/./environment.lock.yml
    status : 1
    message:
      /Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/gateways/logging.py:66: FutureWarning: functools.partial will be a method descriptor in future Python versions; wrap it in staticmethod() if you want to preserve the old behavior
        record.msg = self.TOKEN_REPLACE(record.msg)
      
      # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
      
          Traceback (most recent call last):
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/exception_handler.py", line 18, in __call__
              return func(*args, **kwargs)
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/cli/main.py", line 61, in main_subshell
              exit_code = do_call(args, parser)
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/cli/conda_argparse.py", line 205, in do_call
              result = getattr(module, func_name)(args, parser)
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/notices/core.py", line 132, in wrapper
              return func(*args, **kwargs)
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/cli/main_env_create.py", line 121, in execute
              spec = specs.detect(
                  name=args.name,
                  filename=get_filename(args.file),
                  directory=os.getcwd(),
              )
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/deprecations.py", line 174, in inner
              return func(*args, **kwargs)
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/env/specs/__init__.py", line 62, in detect
              if spec.can_handle():
                 ~~~~~~~~~~~~~~~^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/env/specs/yaml_file.py", line 19, in can_handle
              self._environment = env.from_file(self.filename)
                                  ~~~~~~~~~~~~~^^^^^^^^^^^^^^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/env/env.py", line 173, in from_file
              return from_yaml(yamlstr, filename=filename)
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/env/env.py", line 139, in from_yaml
              data = yaml_safe_load(yamlstr)
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/conda/common/serialize.py", line 44, in yaml_safe_load
              return _yaml_safe().load(string)
                     ~~~~~~~~~~~~~~~~~^^^^^^^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/ruamel/yaml/main.py", line 451, in load
              return constructor.get_single_data()
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/ruamel/yaml/constructor.py", line 114, in get_single_data
              node = self.composer.get_single_node()
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/ruamel/yaml/composer.py", line 71, in get_single_node
              if not self.parser.check_event(StreamEndEvent):
                     ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/ruamel/yaml/parser.py", line 141, in check_event
              self.current_event = self.state()
                                   ~~~~~~~~~~^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/ruamel/yaml/parser.py", line 187, in parse_implicit_document_start
              if not self.scanner.check_token(DirectiveToken, DocumentStartToken, StreamEndToken):
                     ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/ruamel/yaml/scanner.py", line 170, in check_token
              self.fetch_more_tokens()
              ~~~~~~~~~~~~~~~~~~~~~~^^
            File "/Users/mahpa906/.pixi/envs/conda/lib/python3.13/site-packages/ruamel/yaml/scanner.py", line 313, in fetch_more_tokens
              raise ScannerError(
              ...<4 lines>...
              )
          ruamel.yaml.scanner.ScannerError: while scanning for the next token
          found character '@' that cannot start any token
            in "<unicode string>", line 4, column 1:
              @EXPLICIT
              ^ (line: 4)
      
      `$ /Users/mahpa906/.pixi/envs/conda/bin/conda env create --prefix /Users/mahpa906/Sandbox/Nextflow_sandbox/work/conda/env-b29a87e26dc74e81-fec495b0e47379d85cd966c886d2a4d3 --file /Users/mahpa906/Sandbox/Nextflow_sandbox/./environment.lock.yml`
      
        environment variables:
           BUNDLED_DEBUGPY_PATH=/Users/mahpa906/.vscode/extensions/ms-python.debugpy-2025.6.0-darwin-
                                arm64/bundled/libs/debugpy
                       CIO_TEST=<not set>
              CONDA_DEFAULT_ENV=Nextflow_sandbox
                   CONDA_PREFIX=/Users/mahpa906/.pixi/envs/conda
                     CONDA_ROOT=/Users/mahpa906/.pixi/envs/conda
                 CURL_CA_BUNDLE=<not set>
           JAVA_LD_LIBRARY_PATH=/Users/mahpa906/Sandbox/Nextflow_sandbox/.pixi/envs/default/lib/jvm/li
                                b/server
                     LD_PRELOAD=<not set>
                           PATH=/Users/mahpa906/.pixi/envs/conda/bin:/Users/mahpa906/Sandbox/Nextflow_
                                sandbox/.pixi/envs/default/bin:/Users/mahpa906/.vscode/extensions/ms-
                                python.python-2025.4.0-darwin-arm64/python_files/deactivate/zsh:/Users
                                /mahpa906/Sandbox/Nextflow_sandbox/.pixi/envs/default/bin:/usr/local/b
                                in:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/ru
                                n/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/va
                                r/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/
                                run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleintern
                                al/bin:/opt/X11/bin:/opt/podman/bin:/Applications/quarto/bin:/Users/ma
                                hpa906/.vscode/extensions/ms-python.python-2025.4.0-darwin-arm64/pytho
                                n_files/deactivate/zsh:/Users/mahpa906/Sandbox/Nextflow_sandbox/.pixi/
                                envs/default/bin:/Users/mahpa906/.sdkman/candidates/java/current/bin:/
                                Users/mahpa906/.pixi/bin:/Users/mahpa906/.pixi/bin
             REQUESTS_CA_BUNDLE=<not set>
                  SSL_CERT_FILE=<not set>
      
           active environment : base
          active env location : /Users/mahpa906/.pixi/envs/conda
             user config file : /Users/mahpa906/.condarc
       populated config files : /Users/mahpa906/.condarc
                conda version : 25.1.1
          conda-build version : not installed
               python version : 3.13.3.final.0
                       solver : libmamba (default)
             virtual packages : __archspec=1=m1
                                __conda=25.1.1=0
                                __osx=15.4.1=0
                                __unix=0=0
             base environment : /Users/mahpa906/.pixi/envs/conda  (read only)
            conda av data dir : /Users/mahpa906/.pixi/envs/conda/etc/conda
        conda av metadata url : None
                 channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                                https://conda.anaconda.org/conda-forge/noarch
                                https://conda.anaconda.org/bioconda/osx-arm64
                                https://conda.anaconda.org/bioconda/noarch
                package cache : /Users/mahpa906/.pixi/envs/conda/pkgs
                                /Users/mahpa906/.conda/pkgs
             envs directories : /Users/mahpa906/.conda/envs
                                /Users/mahpa906/.pixi/envs/conda/envs
                     platform : osx-arm64
                   user-agent : conda/25.1.1 requests/2.32.3 CPython/3.13.3 Darwin/24.4.0 OSX/15.4.1 solver/libmamba conda-libmamba-solver/25.4.0 libmambapy/2.1.1
                      UID:GID : 501:20
                   netrc file : None
                 offline mode : False
      
      
      An unexpected error has occurred. Conda has prepared the above report.
      If you suspect this error is being caused by a malfunctioning plugin,
      consider using the --no-plugins option to turn off plugins.
      
      Example: conda --no-plugins install <package>
      
      Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
      the command line to run the command without plugins enabled.
      
      Example: CONDA_NO_PLUGINS=true conda install <package>



 -- Check '.nextflow.log' file for details
a
Yeah, so lockfiles aren’t a yaml it seems - I just went and looked them up on wave build page. So I think maybe you need
.txt
m
Looks like that was it
Copy code
[b8/ee7c6d] TASK | 1 of 1 ✔
1

Completed at: 13-May-2025 10:01:30
Duration    : 1m 30s
CPU hours   : (a few seconds)
Succeeded   : 1
a
And of course had I followed links in the original issue I would have seen that 😂 https://github.com/nextflow-io/nextflow/issues/5453
But yeah the answer is currently lockfiles must end
.txt
and must be a real, local (not http) path
m
Good to know. Thank you
j
I heard the summons, but looks like things are in hand. Lock files FTW!
p
Yeah sorry - should have called that, there's the thing with the file extension 😅
We should probably flag it in the docs at least. @Chris Hakkaart when you see this you mind swinging in an admonition to the NF docs that conda lock files need a txt extension please?
Or I can once my power is back on shakefist
j
I'd forgotten that I'd raised that issue after my own testing
m
I had forgotten all about the lock files
😱 1
m
I've been running into a lot of issues running a snakemake pipeline because of package drift. That's why it's on my mind
j
It's been on my mind for ages, I pestered hard to get those lock files produced by Wave for exactly this reason. I'm going to be very excited when we get that infra in place so it's a no-brainer to run in prod with locked-down conda envs.
💯 2
p
Yup, plan in the blog post is to use them by default for
-profile conda
🎉 2
ie. You'll need to explicitly use
-profile conda_env
to use the current environment.yml setup