Hi everyone! I'm setting up nf-core/sarek (v3.6.1) for tumor-only somatic variant calling (Mutect2) on Azure Batch, and I'd really appreciate any guidance from folks who have successfully run this setup. My setup: - nf-core/sarek 3.5.1 (stock remote, not a local clone) - Nextflow 24.10.5 - Azure Batch with a pre-existing pool (Standard_DS3_v2, Ubuntu HPC 24.04, dockerCompatible) - Authentication: Managed identity only (no account keys) - Storage: Azure Blob — separate containers for reference (GRCh37), input FASTQs, work dir, and results - All paths use az:// (e.g., az://nextflow-reference/reference.fa) - Reference: GRCh37 (Ensembl), af-only-gnomad, skipping BQSR for initial validation - Test data: 100 MB paired tumor FASTQ Where I'm stuck: Our pool had a start-task (blobfuse2 mount script) that fails with

CommandProgramNotFound

— the node can't download the script from blob, so it goes to

startTaskFailed

and no Nextflow tasks can execute. I now understand that Nextflow's azurebatch executor uses az:// paths + azcopy for data movement (not blobfuse2 mounts), so the start-task isn't needed for the pipeline itself. But I want to make sure I'm not missing something before removing it. My questions: 1. Has anyone successfully run nf-core/sarek on Azure Batch with managed identity auth (no account keys)? If so, could you share your nextflow.config structure? 2. Does your pool use a start-task, or does it boot clean with just dockerCompatible? What does your pool configuration look like? 3. Any gotchas with az:// paths for --fasta, --bwa, --germline_resource etc.? Do they resolve correctly on the compute nodes? 4. For those who ran sarek 3.5.1 specifically — did you need to pin NXF_VER to avoid the "Variable declarations cannot be mixed with config statements" error on newer Nextflow versions? If anyone has a working config or a step-by-step doc they followed, I'd be incredibly grateful. Happy to share what I've learned so far as well. Thanks in advance!

@Adam Talbot could you help by any chance?

> blobfuse2 mount script What does that mean? blobfuse2 doesn't work with Nextflow

Thank you, Adam, for the clarification. We removed the blobfuse2 dependency and are now able to access files successfully from Azure Blob Storage using the

az://

paths. At the moment, we have moved past the storage access issue and are encountering an error during the BWA alignment step, which we are currently investigating. I have one additional question regarding data mirroring. While attempting to mirror reference resources from the iGenomes/AWS S3 bucket to Azure Blob Storage, we came across documentation that references the environment variables: AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY

yes?

Thanks, Adam. For mirroring the required iGenomes and annotation-cache resources to Azure Blob Storage, do we need AWS credentials, or are those S3 buckets publicly accessible for download? We are planning to mirror GRCh37 and GRCh38 resources locally for our deployment. What's the recommended method for mirroring only the required GRCh37 and GRCh38 resources from iGenomes into Azure Blob Storage? aws s3 sync, azcopy, or another approach?

iGenomes is hosted on AWS S3: nf-co.re/usage/reference_genomes#… If you want to import them to Azure you can copy them with whatever technique is the most suitable for your use case. I tend to use rclone, but I imagine azcopy would also work.

Hi team, We have mirrored the required reference files (FASTA, BWA indexes, and GATKBundle for both GRCh37 and GRCh38) from iGenomes into our Azure Blob Storage and are running nf-core/sarek v3.5.1 on Azure Batch. FastQC and Fastp are able to fetch the FASTQ files from Blob Storage and execute successfully. However, during the alignment step, Sarek appears to be generating BWA index files instead of using the existing index files already present in our Blob container. For example, the following BWA index files already exist in Azure Blob:

human_g1k_v37_decoy.fasta.amb
human_g1k_v37_decoy.fasta.ann
human_g1k_v37_decoy.fasta.bwt
human_g1k_v37_decoy.fasta.pac
<http://human_g1k_v37_decoy.fasta.sa|human_g1k_v37_decoy.fasta.sa>

We are running Sarek with: nextflow run nf-core/sarek -r 3.5.1 -c nextflow.config \ --input samplesheet.csv --outdir 'az://nextflow-work/results/run5' \ --tools mutect2 --aligner bwa-mem --trim_fastq \ --skip_tools baserecalibrator \ --igenomes_ignore --genome null \ --fasta 'az://nextflow-work/ref/human_g1k_v37_decoy.fasta' \ --fasta_fai 'az://nextflow-work/ref/human_g1k_v37_decoy.fasta.fai' \ --dict 'az://nextflow-work/ref/human_g1k_v37_decoy.dict' \ --bwa 'az://nextflow-work/ref/bwa' \ --germline_resource 'az://nextflow-work/ref/af-only-gnomad.raw.sites.vcf.gz' \ --germline_resource_tbi 'az://nextflow-work/ref/af-only-gnomad.raw.sites.vcf.gz.tbi' #GRCh38 rclone sync s3anon:ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta azblob:genomics/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta -P rclone sync s3anon:ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/BWAIndex azblob:genomics/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/BWAIndex -P rclone sync s3anon:ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/GATKBundle azblob:genomics/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/GATKBundle -P # GRCh37 rclone sync s3anon:ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh37/Sequence/WholeGenomeFasta azblob:genomics/igenomes/Homo_sapiens/GATK/GRCh37/Sequence/WholeGenomeFasta -P rclone sync s3anon:ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh37/Sequence/BWAIndex azblob:genomics/igenomes/Homo_sapiens/GATK/GRCh37/Sequence/BWAIndex -P rclone sync s3anon:ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh37/Annotation/GATKBundle azblob:genomics/igenomes/Homo_sapiens/GATK/GRCh37/Annotation/GATKBundle -P Questions: 1. Is Sarek expected to recreate the BWA index for every new run/batch, even when the index files already exist? 2. If we run another batch a few days later, will the indexing process be repeated? 3. Is there a specific directory structure or parameter required for Sarek to reuse prebuilt BWA indexes stored in Azure Blob Storage? 4. Has anyone successfully configured Sarek to consume existing BWA indexes from remote object storage (Azure Blob/S3) instead of generating them locally? Any guidance would be greatly appreciated. Thanks!

You should provide it with --bwa https://nf-co.re/sarek/3.9.0/parameters/#bwa

We tried --bwa but still it is creating the index file

Could you share a log file so I can assess this issue?

command.sh,command.run,command.log,command.err

That's for bwa_mem, that's expected

Thanks, Maxime. Just to clarify — does the log indicate that the BWA indexing step completed successfully and that the current failure is occurring during the bwa mem alignment step? If so, could you advise which log file or specific section we should focus on to identify why the alignment step is failing?

This log only informed me about the BWA_MEM process, there is indeed a failure, not sure where it's coming from. I cannot say anything about BWA_INDEX given that log

The command I have given is wrong?

What was the command you launched sarek with?

nextflow run nf-core/sarek -r 3.5.1 -c nextflow.config --input samplesheet.csv --outdir 'az://nextflow-work/results/run6' --tools mutect2 --aligner bwa-mem --trim_fastq --skip_tools baserecalibrator --igenomes_ignore --genome null --fasta 'az://nextflow-work/ref/human_g1k_v37_decoy.fasta' --fasta_fai 'az://nextflow-work/ref/human_g1k_v37_decoy.fasta.fai' --dict 'az://nextflow-work/ref/human_g1k_v37_decoy.dict' --germline_resource 'az://nextflow-work/ref/af-only-gnomad.raw.sites.vcf.gz' --germline_resource_tbi 'az://nextflow-work/ref/af-only-gnomad.raw.sites.vcf.gz.tbi'

ok, so for me, to get the bwa indexes, you also need to add

--bwa '<az://nextflow-work/ref/bwa_index>'