Hi all, I'm running the pipeline for the first tim...
# methylseq
y
Hi all, I'm running the pipeline for the first time. I got the following error:
Copy code
WARN: Unable to get file attributes file: <s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa> -- Cause: software.amazon.awssdk.services.s3.model.S3Exception: (Service: S3, Status Code: 403, Request ID: null) (SDK Attempt Count: 1)
WARN: Unable to get file attributes file: <s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Sequence/BismarkIndex> -- Cause: software.amazon.awssdk.services.s3.model.S3Exception: (Service: S3, Status Code: 403, Request ID: null) (SDK Attempt Count: 1)
I saw https://nfcore.slack.com/archives/CP3RJSMF0/p1721641186374589 but I'm not sure I understand and if I need to install the standalone aws . Could it be a problem with our proxy ? Thanks
👀 1
m
the files do exist, so I'm guessing more a problem with an anonymous access to it
y
@Maxime Garcia I'm not sure I understand. Is there anything I can do on my side (proxy... ) ?
m
you can run rnaseq or sarek on your side?
y
@Maxime Garcia same error for Sarek:
Copy code
WARN: Unable to get file attributes file: <s3://ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta/Homo_sapiens_assembly38.fasta> -- Cause: software.amazon.awssdk.services.s3.model.S3Exception: (Service: S3, Status Code: 403, Request ID: null) (SDK Attempt Count: 1)
WARN: Unable to get file attributes file: <s3://ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/GATKBundle/dbsnp_146.hg38.vcf.gz> -- Cause: software.amazon.awssdk.services.s3.model.S3Exception: (Service: S3, Status Code: 403, Request ID: null) (SDK Attempt Count: 1)
ERROR ~ (Service: S3, Status Code: 403, Request ID: null) (SDK Attempt Count: 1)
m
Yeah. I would recommend downloading the data beforehand and using the igenomes_base to point to the right direction
✅ 1
y
@Maxime Garcia ok merci/thanks
p
I think this is the thing where you need no-sign-request to access the public data in s3 iGenomes
And if you have your own AWS credentials set up, that's effectively impossible
So it tries to authenticate to the iGenomes bucket with your credentials, which fails
As @Maxime Garcia says, the easy fix is to make a copy of the reference data 😕
I still hope for a longer-term fix at some point. We were discussing it not long ago in the Nextflow team.
If you're using Seqera / Fusion, I guess you should be able to mount the iGenomes bucket and then point to that as a local path. That's probably fine, and avoids the copying step..
m
If I remember well, the issue Pierre is facing is that he's running on a machine behind a proxy
✅ 1
👍🏻 1
y
update: yes, that was the proxy.
👍🏻 1
➕ 1