Hi everyone, We are deploying nf-core/Sarek in a r...
# sarek
a
Hi everyone, We are deploying nf-core/Sarek in a restricted enterprise environment and are currently working with our infrastructure team on outbound whitelisting requirements. Based on our investigation, we have identified the following domains/endpoints: github.com codeload.github.com raw.githubusercontent.com objects.githubusercontent.com registry.nextflow.io ngi-igenomes.s3.eu-west-1.amazonaws.com s3.eu-west-1.amazonaws.com community.wave.seqera.io community-cr-prod.seqera.io wave.seqera.io quay.io cdn.quay.io registry-1.docker.io auth.docker.io production.cloudflare.docker.com Could anyone confirm whether this list is sufficient for a standard nf-core/Sarek deployment, or if there are additional domains/endpoints that should also be considered? Thanks!
m
HI @a.br.ravichandran, thanks a lot for your question. I think you're missing the annotation-cache s3 bucket from your list. I am also unsure you need to whitelist all these domains/endpoints. As replied previously, We have a dedicated resource for this particular topic on our website: https://nf-co.re/usage/offline The nf-core tools CLI has a dedicated subcommand to download most dependencies as Singularity/Apptainer images: https://nf-co.re/docs/nf-core-tools/cli/pipelines/download For the necessary data, I recommend pre-downloading the files from ngi-igenomes. Follow the genome key to get the list of files from https://github.com/nf-core/sarek/blob/master/conf/igenomes.config For annotation, I recommend https://annotation-cache.github.io/ that has built-in support in Sarek. Similarly, you can pre-download these files. From what I understand you do not run in a offline setup similarly to how we would run things in a typical academic setting. In that case, what we would do would be download all necessary dependencies and then transfer all to a closed environment. For me, for an offline usage, I would. Pre-download nextflow and all necessary plugins. (versions can be found in the root nextflow.config). If you plan to use docker over singularity/apptainer, I would recommend having a custom registry, and overriding via a custom config file all container declarations and pointing them all to your own containers. I would recommend to mirror the s3 buckets locally (no need to have all files, just the one you need for the analysis). so in sarek case, annotation-cache on top of igenomes. I would love to take this as an opportunity to improve the documentation we have on running offline.