Ramon Massoni
03/24/2026, 11:25 PMBWA_MEM never receives any tasks when the pipeline builds the BWA index from scratch (i.e., when params.bwa_index is null/not provided). The pipeline completes "successfully" but skips alignment and everything downstream.
Environment:
• Nextflow 24.04.2
• nf-core/atacseq v2.1.2
• Singularity (HPC cluster, NYGC profile)
• --genome GRCh38 (iGenomes, S3-backed references)
What happens:
1. PREPARE_GENOME:BWA_INDEX builds the index from the FASTA — completes successfully (exit 0), index files are correct.
2. FASTQ_FASTQC_UMITOOLS_TRIMGALORE completes — 18/18 FASTQ pairs processed.
FASTQ_ALIGN_BWA:BWA_MEM shows - (0 tasks submitted). Never fires.
All downstream processes (samtools, picard, MACS2, deeptools) also show -.
1. Pipeline reports "completed successfully" because processes with 0 tasks are treated as complete.
Reproduction:
This happens on a completely fresh run (clean work directory, no -resume cache). I confirmed by deleting the work/ and .nextflow/ directories and re-running. The Nextflow log confirms: "It appears you have never run this project before -- Option -resume is ignored".
Root cause (suspected):
In subworkflows/local/prepare_genome.nf, the BWA index channel is created differently depending on whether a pre-built index is provided:
• Pre-built index provided (params.bwa_index is set):
•
• ch_bwa_index = [ [:], file(params.bwa_index) ]
• This creates a plain Groovy list, which Nextflow implicitly treats as a value channel. Value channels broadcast to all items in the reads queue channel → BWA_MEM gets 18 tasks. ✅
• Index built from FASTA (params.bwa_index is null):
ch_bwa_index = BWA_INDEX ( ch_fasta.map { [ [:], it ] } ).index
This is a queue channel (process output). Queue channels pair one-to-one with other queue channels. Since the reads channel has 18 items and the index channel has 1 item, at most 1 BWA_MEM task could be created — but in practice 0 tasks fire, and the pipeline silently completes. ✘
In workflows/atacseq.nf, the index is passed directly without .collect() or .first() to convert it to a value channel:
FASTQ_ALIGN_BWA (
FASTQ_FASTQC_UMITOOLS_TRIMGALORE.out.reads,
PREPARE_GENOME.out.bwa_index, // <-- queue channel when built, value channel when pre-built
false,
PREPARE_GENOME.out.fasta.map { [ [:], it ] }
)
Suggested fix:
Convert the queue channel to a value channel before passing to FASTQ_ALIGN_BWA, e.g.:
PREPARE_GENOME.out.bwa_index.collect()
// or
PREPARE_GENOME.out.bwa_index.first()
Why this matters:
On HPC clusters without direct S3 access (or where S3 directory paths aren't supported), the default iGenomes bwa_index path (<s3://ngi-igenomes/.../BWAIndex/version0.6.0/>) can't be resolved. Setting params.bwa_index = null to force local index building is the expected workaround, but it triggers this channel-type mismatch.
Context:
• The same pipeline (v2.1.2) with --genome mm10 works fine because mm10 iGenomes does not include a pre-built BWA index, so the "build from FASTA" path is always taken. (I confirmed this from a previous successful run with the same Nextflow version.)
• nf-core/cutandrun v3.2.2 does not have this issue — its Bowtie2 index handling resolves S3 paths correctly.
Happy to provide full logs or .nextflow.log if helpful. Thanks!Maxime Garcia
03/25/2026, 8:51 AMJose Espinosa-Carrasco
03/25/2026, 10:44 AMJonathan Manning
03/25/2026, 10:48 AMMaxime Garcia
03/25/2026, 10:50 AMJonathan Manning
03/25/2026, 10:54 AMJonathan Manning
03/25/2026, 10:55 AMJose Espinosa-Carrasco
03/25/2026, 10:56 AMcheckIfExists on this line fix the issue: ch_bwa_index = [[:], file(bwa_index)]
?Jose Espinosa-Carrasco
03/25/2026, 10:57 AMJose Espinosa-Carrasco
03/25/2026, 10:58 AMJonathan Manning
03/25/2026, 11:01 AMJose Espinosa-Carrasco
03/25/2026, 11:01 AMJose Espinosa-Carrasco
03/25/2026, 12:04 PMs3_index_validation @Ramon Massoni?
To double-check that if fixes the issueRamon Massoni
03/25/2026, 1:27 PMJose Espinosa-Carrasco
03/26/2026, 4:15 PMJose Espinosa-Carrasco
03/30/2026, 2:39 PMRamon Massoni
03/30/2026, 4:20 PMRamon Massoni
03/30/2026, 4:28 PMnextflow run nf-core/atacseq \
-r s3_index_validation \
-resume \
-profile nygc \
--input "$SAMPLESHEET" \
--outdir "$OUTDIR" \
--genome GRCh38 \
-c "${PROJECT_DIR}/epigenome_cell_lines/scripts/1-mapping/ATAC/override_igenomes.config" \
--read_length 50 \
--narrow_peak \
--save_reference
will keep you postedRamon Massoni
03/30/2026, 4:35 PMThe s3_index_validation branch fails on Nextflow 24.04.2 with a compilation error:
Module compilation error
- file : workflows/atacseq.nf
- cause: Variable `channel` already defined in the process scope @ line 118, column 26.
def topic_versions = channel.topic("versions")
^
The PR changed Channel (uppercase) to channel (lowercase) throughout <http://prepare_genome.nf|prepare_genome.nf> for "strict mode syntax", but workflows/atacseq.nf still uses channel as a local variable name. Nextflow 24.04.2 interprets lowercase channel as a variable reference, not the Nextflow Channel factory class, causing a name collision at line 118.
Bottom line: The branch can't even launch on Nextflow 24.04.2, so it's untestable — and separately, it still doesn't address the queue-vs-value channel root cause for BWA_MEM receiving 0 tasksJose Espinosa-Carrasco
03/30/2026, 4:52 PMRamon Massoni
03/30/2026, 5:01 PMJose Espinosa-Carrasco
03/30/2026, 5:16 PMJose Espinosa-Carrasco
03/30/2026, 5:16 PMRamon Massoni
03/30/2026, 5:29 PMRamon Massoni
03/30/2026, 5:33 PM[2026-03-30 13:29:42] job START on ne1dc3-015
N E X T F L O W ~ version 24.04.2
WARN: Invalid config manifest attribute `contributors`
WARN: It appears you have never run this project before -- Option `-resume` is ignored
Launching `<https://github.com/nf-core/atacseq>` [trusting_wing] DSL2 - revision: e2518260cd [s3_index_validation]
WARN: Nextflow self-contained distribution allows only core plugins -- User config plugins will be ignored: nf-schema@2.5.1
WARN: Invalid config manifest attribute `contributors`
ERROR ~ Module compilation error
- file : /gpfs/commons/home/rmassoni/.nextflow/assets/nf-core/atacseq/./workflows/atacseq.nf
- cause: Variable `channel` already defined in the process scope @ line 118, column 26.
def topic_versions = channel.topic("versions")
^
1 error
-- Check script '/gpfs/commons/home/rmassoni/.nextflow/assets/nf-core/atacseq/main.nf' at line: 37 or see '.nextflow.log' file for more details
[2026-03-30 13:30:28] job ENDRamon Massoni
03/30/2026, 5:34 PMAs expected — they only reverted channel back to Channel in <http://prepare_genome.nf|prepare_genome.nf>, but not in workflows/atacseq.nf where the error actually is (line 118: channel.topic("versions")).
And critically — looking at the BWA index section, the revert still shows:
ch_bwa_index = BWA_INDEX ( ch_fasta.map { item -> [ [:], item ] } ).index
No .collect() or .first() — the queue/value channel bug is still there.
Tell them:
Two issues with the latest commit (e251826):
Compilation still fails — workflows/atacseq.nf line 118 still has lowercase channel.topic("versions"). The revert only hit <http://prepare_genome.nf|prepare_genome.nf> but not <http://atacseq.nf|atacseq.nf>.
The original bug is still unfixed — the BWA index path when building from scratch (BWA_INDEX(...).index) is still a queue channel with no .collect() or .first(). Even if the compilation error is resolved, BWA_MEM will still receive 0 tasks.Jose Espinosa-Carrasco
03/31/2026, 2:08 PMRamon Massoni
03/31/2026, 9:03 PMRamon Massoni
03/31/2026, 9:13 PMmodule load Nextflow/25.10.0
seems to be running well so farRamon Massoni
04/02/2026, 5:23 PMexecutor > slurm (45)
[cb/639c94] NFC…GENOME:GTF2BED (genes.gtf) | 1 of 1 ✔
[4c/4040ab] NFC…PREPARE_GENOME:TSS_EXTRACT | 1 of 1 ✔
[c7/7a66b1] NFC…_GETCHROMSIZES (genome.fa) | 1 of 1 ✔
[3f/72c7c0] NFC…_AUTOSOMES (genome.fa.fai) | 1 of 1 ✔
[1b/bc06c7] NFC…_REGIONS (genome.fa.sizes) | 1 of 1 ✔
[49/175f61] NFC…NOME:BWA_INDEX (genome.fa) | 1 of 1 ✔
[63/7e1891] NFC…ECK (atac_samplesheet.csv) | 1 of 1 ✔
[3e/f6c83e] NFC…:FASTQC (U-RT1_WT_REP2_T1) | 18 of 18 ✔
[e8/1a9fe5] NFC…MGALORE (U-RT1_WT_REP3_T1) | 18 of 18 ✔
[- ] NFC…EQ:FASTQ_ALIGN_BWA:BWA_MEM -
[- ] NFC…ATS_SAMTOOLS:SAMTOOLS_SORT -
[- ] NFC…TS_SAMTOOLS:SAMTOOLS_INDEX -
[- ] NFC…TS_SAMTOOLS:SAMTOOLS_STATS -
[- ] NFC…SAMTOOLS:SAMTOOLS_FLAGSTAT -
[- ] NFC…SAMTOOLS:SAMTOOLS_IDXSTATS -
[- ] NFC…CARD_MERGESAMFILES_LIBRARY -
[- ] NFC…CARD:PICARD_MARKDUPLICATES -
[- ] NFC…ATES_PICARD:SAMTOOLS_INDEX -
[- ] NFC…TS_SAMTOOLS:SAMTOOLS_STATS -
[- ] NFC…SAMTOOLS:SAMTOOLS_FLAGSTAT -
[cb/c7965f] NFCORE_ATACSEQ:ATACSEQ:IGV | 1 of 1 ✔
[64/2ae3cd] NFCORE_ATACSEQ:ATACSEQ:MULTIQC | 1 of 1 ✔
Plus 51 more processes waiting for tasks…
-[nf-core/atacseq] Pipeline completed successfully-
Completed at: 31-Mar-2026 22:18:52
Duration : 1h 9m 1s
CPU hours : 1.6
Succeeded : 45Jose Espinosa-Carrasco
04/07/2026, 5:17 PMs3_index_validation and run the sample test files and it runs the bwa process, I used this command:
NXF_VER=25.10.4 nextflow run <http://main.nf|main.nf> \
--input <https://raw.githubusercontent.com/nf-core/test-datasets/atacseq/samplesheet/v2.0/samplesheet_test.csv> \
--outdir ./results \
--genome GRCh38 \
--read_length 50 \
-profile crg \
-ansi-log false
And here is part of the log:
e4/034087] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T150_SE_REP1_T1)
[f1/d9fb6a] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T150_SE_REP1_T2)
[d0/049e0c] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (OSMOTIC_STRESS_T150_SE_REP1_T2)
[b6/b919b0] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (OSMOTIC_STRESS_T15_PE_REP1_T1)
[6a/520a3c] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T15_PE_REP1_T1)
[f6/a66117] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (OSMOTIC_STRESS_T15_PE_REP1_T2)
[9d/704268] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T15_PE_REP1_T2)
[f7/a4fbe9] Submitted process > NFCORE_ATACSEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES (genome.fa)
[38/80dedb] Submitted process > NFCORE_ATACSEQ:PREPARE_GENOME:GET_AUTOSOMES (genome.fa.fai)
[72/d4bef0] Submitted process > NFCORE_ATACSEQ:PREPARE_GENOME:GENOME_BLACKLIST_REGIONS (genome.fa.sizes)
Staging foreign file: <s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Sequence/BWAIndex/version0.6.0>
[a0/7d2210] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T0_PE_REP1_T1)
[b1/79e881] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T150_SE_REP1_T2)
[e7/04ed4b] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T150_SE_REP1_T1)
[8b/c66467] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T0_PE_REP2_T1)
[52/bba807] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T100_SE_REP2_T1)
[68/4268fb] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T100_SE_REP1_T1)
[19/7c2a42] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T15_PE_REP1_T1)
[16/70e28f] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T15_PE_REP1_T2)