https://nf-co.re logo
Join Slack
Powered by
Is anybody else having container issues with this ...
# funcscan
m
I’m getting
Copy code
Error executing process > 'NFCORE_FUNCSCAN:FUNCSCAN:BGC:ANTISMASH_ANTISMASHDOWNLOADDATABASES' Caused by: Failed to pull singularity image command: singularity pull --name nf-core-antismash-8.0.1--pyhdfd78af_0.img.pulling.1764486797668 <docker://nf-core/antismash:8.0.1--pyhdfd78af_0> > /dev/null status : 255 hint : Try and increase singularity.pullTimeout in the config (current is "1h") message: FATAL: While making image from oci registry: error fetching image to cache: failed to get checksum for <docker://nf-core/antismash:8.0.1--pyhdfd78af_0>: reading manifest 8.0.1--pyhdfd78af_0 in <http://docker.io/nf-core/antismash|docker.io/nf-core/antismash>: errors: denied: requested access to the resource is denied unauthorized: authentication required
I tried logging in and pulling the image directly but I couldn’t still (By the way, do you ever need to login with singularity to pull images with nextflow or does it have a sophisticated way to pull things without needing to?)
Copy code
$ singularity remote login --username mpampuch <docker://docker.io>
Password / Token: 

INFO:    Token stored in /home/pampum/.singularity/remote.yaml

$ singularity pull <docker://nf-core/antismash:8.0.1--pyhdfd78af_0>
FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for <docker://nf-core/antismash:8.0.1--pyhdfd78af_0>: reading manifest 8.0.1--pyhdfd78af_0 in <http://docker.io/nf-core/antismash|docker.io/nf-core/antismash>: errors:
denied: requested access to the resource is denied
unauthorized: authentication required
Also looking at the module syntax (https://github.com/nf-core/funcscan/blob/fa9db018e528ffb5149cdde928e2fa24e7c546fe/modules/nf-core/antismash/antismashdownloaddatabases/main.nf#L5), I’m wondering why the container definition doesn’t seem to have a full url like most other modules I’ve seen:
Copy code
container "nf-core/antismash:8.0.1--pyhdfd78af_0"
Most other nf-core modules I seen look something like this:
Copy code
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
        '<https://depot.galaxyproject.org/singularity/bakta:1.10.4--pyhdfd78af_0>' :
        'biocontainers/bakta:1.10.4--pyhdfd78af_0' }"
I’m wondering if this has anything to do with it potentially
j
The antismash module is different as we cannot use the bioconda/biocontainer for it, due to some odd parts of antiSMASH databsae validation that doesn't really work with containers nicely. Therefore we build our own container which we host on the nf-core quay.io account. But all of these containers are public, so there shouldn't be an issue Why you're having issues pulling it I'm not sure... I'm going to test myself now and see if it works. I'll report back
It worked fine for me...
Copy code
nextflow run ../main.nf -profile test_bgc_pyrodigal,singularity --outdir ./results
<...>
Pulling Singularity image <docker://quay.io/nf-core/antismash:8.0.1--pyhdfd78af_0> [cache /home/james/cache/singularity/quay.io-nf-core-antismash-8.0.1--pyhdfd78af_0.img]
<...>
[8d/593ff5] NFCORE_FUNCSCAN:FUNCSCAN:BGC:ANTISMASH_ANTISMASH (sample_2)                          [100%] 1 of 1 ✔
Maybe quay.io was having problems last night/yesterday?
m
Strange, I tried it again and I am having the same issue. The pipeline almost fails immediately for me
Copy code
Error executing process > 'NFCORE_FUNCSCAN:FUNCSCAN:BGC:ANTISMASH_ANTISMASHDOWNLOADDATABASES'

Caused by:
  Failed to pull singularity image
    command: singularity pull  --name nf-core-antismash-8.0.1--pyhdfd78af_0.img.pulling.1764585730618 <docker://nf-core/antismash:8.0.1--pyhdfd78af_0> > /dev/null
    status : 255
    hint   : Try and increase singularity.pullTimeout in the config (current is "3h")
    message:
      FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for <docker://nf-core/antismash:8.0.1--pyhdfd78af_0>: reading manifest 8.0.1--pyhdfd78af_0 in docker.io/nf-core/antismash: errors:
      denied: requested access to the resource is denied
      unauthorized: authentication required
Command used:
Copy code
nextflow run '<https://github.com/nf-core/funcscan>'
		 -name nice_ritchie
		 -params-file '<https://api.cloud.seqera.io/ephemeral/KEwUjT0LVgBvG_NzdVtozw.json>'
		 -with-tower
		 -r 3.0.0
		 -profile singularity,kaust,test
Full log:
I don’t see 
quay
written anywhere in my logs. Maybe my nextflow pipeline isn’t looking for the container in the right registry. Do I have to configure quay somewhere?
j
I think the 
<http://quay.io|quay.io>
thing is default in Nextflow
And running 
singularity pull  --name nf-core-antismash-8.0.1--pyhdfd78af_0.img.pulling.1764585730618 <docker://nf-core/antismash:8.0.1--pyhdfd78af_0> > /dev/null
fails too or this runs through?
But it's also odd as all the other biocontainers of all other modules are also on 
<http://quay.io|quay.io>
🤔
Hrm wait, none of your own tasks start either by the looks of it...
If you skip the BGC subworkflow, does it then run through OK?
m
I got it working. All I had to do was add
Copy code
singularity.registry = '<http://quay.io|quay.io>'
To my 
nextflow.config
For some reason, all the other container technologies had these in their resolved configuration without me specifying it, but for singularity I had to manually do it. Thanks for the help!
j
Oh lol sorry
We do actually specify that 😅
But I don't understand why you had to manually specify that
It is in the pipelien code..
It should be inherited 🤔
Does 
kaust
override it something?
m
Yeah I don’t know. Maybe I thought maybe my institutional config overwrited it but didn’t look too deep
I can check
https://github.com/nf-core/configs/blob/master/conf/kaust.config#L13 Don’t think so. This is all it says for singularity
Strange
j
Hrm, I wonder if it's because you specifyied 
singularity
in your 
-profile
as it's already et in 
kaust
you shouldn't need it
m
Let me try to run the pipeline again without that
Huh strange
You were right
Copy code
nextflow run '<https://github.com/nf-core/funcscan>'
		 -name ridiculous_curran
		 -params-file '<https://api.cloud.seqera.io/ephemeral/2GQS38P_yNAqAW84C5XIvA.json>'
		 -with-tower
		 -r 3.0.0
		 -profile singularity,kaust,test
Resolved configuration:
Copy code
singularity {
   enabled = true
   autoMounts = true
   pullTimeout = '3 hours'
   libraryDir = '/biocorelab/BIX/resources/singularity/images/'
   cacheDir = '/ibex/user/pampum/.singularity/nf_images/'
}
Without 
singularity
Copy code
nextflow run '<https://github.com/nf-core/funcscan>'
		 -name small_kalman
		 -params-file '<https://api.cloud.seqera.io/ephemeral/b-LGY3gRBye293xjrsSdjA.json>'
		 -with-tower
		 -r 3.0.0
		 -profile kaust,test
Resolved configuration:
Copy code
singularity {
   enabled = true
   autoMounts = true
   pullTimeout = '3 hours'
   libraryDir = '/biocorelab/BIX/resources/singularity/images/'
   cacheDir = '/ibex/user/pampum/.singularity/nf_images/'
   registry = '<http://quay.io|quay.io>'
}
Bizarre. Could this be a Nextflow or Seqera Platform parsing bug?
j
No I think it's an inheritance thing
See a the big red box bottom of this xsection: https://www.nextflow.io/docs/latest/config.html#config-profiles
m
Ah okay thank you 🙏
Open in Slack
PreviousNext
Powered by