https://nf-co.re logo
Join Slack
Powered by
<@U03SLE7RBAS> you tested the new conda lock-files...
# bioconda
p
@Jonathan Manning you tested the new conda lock-files generated by Wave, right? @mashehu is having difficulties getting them to work with the new Seqera Containers config stuff, details in 🧵
m
*expanding thread for @Phil Ewels * We have a trial setup with fully converted modules in our testpipeline repo https://github.com/nf-core/testpipeline/tree/seqera-containers-conversion-manual running it with 
-profile test,conda
crashes with
Copy code
Creating env using conda: <https://wave.seqera.io/v1alpha1/builds/bd-af7a5314d5015c29_1/condalock> [cache /Users/mitochondrium/testpipeline/work/conda/env-42a777122dcd43d94ef8950adbf39e15]
-[nf-core/testpipeline] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_TESTPIPELINE:TESTPIPELINE:FASTQC (1)'

Caused by:
  Failed to create Conda environment
    command: conda create --yes --quiet --prefix /Users/mitochondrium/testpipeline/work/conda/env-42a777122dcd43d94ef8950adbf39e15 -c conda-forge -c bioconda <https://wave.seqera.io/v1alpha1/builds/bd-af7a5314d5015c29_1/condalock>
    status : 1
    message:
      2 channel Terms of Service accepted
      Retrieving notices: ...working... done
      Channels:
       - conda-forge
       - bioconda
       - defaults
      Platform: osx-arm64
      Collecting package metadata (repodata.json): ...working... done
      Solving environment: ...working... failed

      PackagesNotFoundError: The following packages are not available from current channels:

        - //wave.seqera.io/v1alpha1/builds/bd-af7a5314d5015c29_1/condalock

      Current channels:

        - <https://conda.anaconda.org/conda-forge>
        - <https://conda.anaconda.org/bioconda>
        - defaults

      To search for alternate channels that may provide the conda package you're
      looking for, navigate to

          <https://anaconda.org>

      and use the search bar at the top of the page.
🙏🏻 1
p
@mashehu you're definitely running on the right CPU arch, right? lock files are specific to architectures, environment files are not
quick look 1
m
j
Yep, I definitely got them working (that was when I found that remote issue Phil), but I haven't tried it recently. Lemme have a play
🙏🏻 1
Is this incorporating Paolo's remote + extensions fix @Phil Ewels?
Last time I tried the files had to be local and named with a .txt
p
Not sure. What version of Nextflow were you using @mashehu ?
m
25.10.2
p
But we should be saving local as a txt anyway, either way
No reason not to, gives best backwards compatibility
j
https://github.com/nextflow-io/nextflow/pull/6713 fixed the extension thing, but 
<https://wave.seqera.io/v1alpha1/builds/bd-af7a5314d5015c29_1/condalock>
is probably still not a goer. @mashehu you might have to store the lockfiles on the repo next to the environment ymls and name them .txt for now. (ref)
p
Fix is only on master currently
this 1
Yup, not just for now - we had said that we wanted to store them alongside the environment.yml file anyway instead of relying on a remote service.
And if saving them there's no reason not to use .txt
m
and point the config to the local file? do you have an example pipeline where i can see it?
p
Sorry, I should have spotted this sooner. Didn't look at the log close enough to spot the URL
Yeah, should work the same as when we specify a environment.yml file
j
I never implemented it in anger, you'll be the first @mashehu, but I'm moderately confident it will work based on testing 👀
this is fine fire 2
m
good news: it worked for non-arm conda lock files! 🎉
🎉 2
bad news: i got a non working conda lock file for fastqc in the arm lock file https://wave.seqera.io/v1alpha1/builds/bd-d56b505a93aef38a_1/condalock
j
what was the failure?
m
Copy code
Failed to create Conda environment
    command: conda create --yes --quiet --prefix /Users/mitochondrium/testpipeline/work/conda/env-b57361e2e8f0abebde6131114b961e3c --file /Users/mitochondrium/testpipeline/modules/nf-core/fastqc/.conda/environment-arm64.txt
    status : 1
    message:
      2 channel Terms of Service accepted

      InvalidMatchSpec: Invalid spec '+ echo << CONDA_LOCK_END': Invalid version 'echo<<CONDA_LOCK_END': invalid character(s)



 -- Check '.nextflow.log' file for details
j
Oh yeah, I see the crap:
Copy code
<https://conda.anaconda.org/conda-forge/noarch/font-ttf-dejavu-sans-mono-2.37-hab24e00_0.tar.bz2#0c96522c6bdaed4b1566d11387caaf45>
<https://conda.anaconda.org/conda-forge/noarch/font-ttf-inconsolata-3.000-h77eed37_0.tar.bz2#34893075a5c9e55cdafac56607368fc6>
<https://conda.anaconda.org/conda-forge/noarch/font-ttf-source-code-pro-2.038-h77eed37_0.tar.bz2#4d59c254e01d9cde7957100457e2d5fb>
+ echo << CONDA_LOCK_END
+ micromamba clean -a -y
<https://conda.anaconda.org/conda-forge/noarch/font-ttf-ubuntu-0.83-h77eed37_3.conda#49023d73832ef61042f6a237cb2687e7>
<https://conda.anaconda.org/conda-forge/noarch/fonts-conda-forge-1-0.tar.bz2#f766549260d6815b0c52253f1fb1bb29>
<https://conda.anaconda.org/conda-forge/noarch/fonts-conda-ecosystem-1-0.tar.bz2#fee5683a3f04bd15cbd8318b096a27ab>
I'll report to the team
m
thanks, the one for multiqc is fine
j
Could you confirm for me how the wave build was triggered?
m
okay, I reran the commands now and got a working arm lock file
🎉 1
j
Glad to hear, it's still being discussed internally
👍🏻 1
They removed the bad build, and only found a few other similar ones from the last few months, so fingers crossed it's not a common issue. We don't have a way to force rebuild right now, so they're looking into a better workaround/ mitigation in case it does recurr.
2 Views
Open in Slack
PreviousNext
Powered by