Bug report: nf-core/atacseq v2.1.2 — BWA_MEM silen...
# atacseq
r
Bug report: nf-core/atacseq v2.1.2 — BWA_MEM silently receives 0 tasks when BWA index is built from FASTA (no pre-built index provided) Hi all, I'm running into an issue with nf-core/atacseq v2.1.2 where
BWA_MEM
never receives any tasks when the pipeline builds the BWA index from scratch (i.e., when
params.bwa_index
is null/not provided). The pipeline completes "successfully" but skips alignment and everything downstream. Environment: • Nextflow 24.04.2 • nf-core/atacseq v2.1.2 • Singularity (HPC cluster, NYGC profile) •
--genome GRCh38
(iGenomes, S3-backed references) What happens: 1.
PREPARE_GENOME:BWA_INDEX
builds the index from the FASTA — completes successfully (exit 0), index files are correct. 2.
FASTQ_FASTQC_UMITOOLS_TRIMGALORE
completes — 18/18 FASTQ pairs processed.
FASTQ_ALIGN_BWA:BWA_MEM
shows
-
(0 tasks submitted). Never fires. All downstream processes (samtools, picard, MACS2, deeptools) also show
-
. 1. Pipeline reports "completed successfully" because processes with 0 tasks are treated as complete. Reproduction: This happens on a completely fresh run (clean work directory, no
-resume
cache). I confirmed by deleting the
work/
and
.nextflow/
directories and re-running. The Nextflow log confirms:
"It appears you have never run this project before -- Option -resume is ignored"
. Root cause (suspected): In
subworkflows/local/prepare_genome.nf
, the BWA index channel is created differently depending on whether a pre-built index is provided: • Pre-built index provided (
params.bwa_index
is set): • •
ch_bwa_index = [ [:], file(params.bwa_index) ]
• This creates a plain Groovy list, which Nextflow implicitly treats as a value channel. Value channels broadcast to all items in the reads queue channel → BWA_MEM gets 18 tasks. ✅ • Index built from FASTA (
params.bwa_index
is null):
Copy code
ch_bwa_index = BWA_INDEX ( ch_fasta.map { [ [:], it ] } ).index
This is a queue channel (process output). Queue channels pair one-to-one with other queue channels. Since the reads channel has 18 items and the index channel has 1 item, at most 1 BWA_MEM task could be created — but in practice 0 tasks fire, and the pipeline silently completes. ✘ In
workflows/atacseq.nf
, the index is passed directly without
.collect()
or
.first()
to convert it to a value channel:
Copy code
FASTQ_ALIGN_BWA (
    FASTQ_FASTQC_UMITOOLS_TRIMGALORE.out.reads,
    PREPARE_GENOME.out.bwa_index,   // <-- queue channel when built, value channel when pre-built
    false,
    PREPARE_GENOME.out.fasta.map { [ [:], it ] }
)
Suggested fix: Convert the queue channel to a value channel before passing to
FASTQ_ALIGN_BWA
, e.g.:
Copy code
PREPARE_GENOME.out.bwa_index.collect()
// or
PREPARE_GENOME.out.bwa_index.first()
Why this matters: On HPC clusters without direct S3 access (or where S3 directory paths aren't supported), the default iGenomes
bwa_index
path (
<s3://ngi-igenomes/.../BWAIndex/version0.6.0/>
) can't be resolved. Setting
params.bwa_index = null
to force local index building is the expected workaround, but it triggers this channel-type mismatch. Context: • The same pipeline (v2.1.2) with
--genome mm10
works fine because mm10 iGenomes does not include a pre-built BWA index, so the "build from FASTA" path is always taken. (I confirmed this from a previous successful run with the same Nextflow version.) • nf-core/cutandrun v3.2.2 does not have this issue — its Bowtie2 index handling resolves S3 paths correctly. Happy to provide full logs or
.nextflow.log
if helpful. Thanks!
m
@Jonathan Manning Is this the issue we had?
j
Thanks for reporting @Ramon Massoni
j
sorry Maxime, not sure what you were referring to
m
the s3 can't access bucket issue with rnaseq/sarek
j
This thing was kinda similar: https://github.com/nf-core/rnaseq/pull/1716 (but not an S3 thing)
This is probably what you mean: https://github.com/nf-core/rnaseq/pull/1745/changes ... but this is different, Jose is on the case 🙂
🙏 1
j
Should a
checkIfExists
on this line fix the issue:
ch_bwa_index = [[:], file(bwa_index)]
?
I mean if the problem is that the s3 path can’t be resolved, the bwa_index has been provided, but as we don’t check whether it exists it silently does not run anything downstream
j
Yes, I think so
j
I will also add what you have done here
👍 1
PR here in case someone fancies reviewing it 😃 In any case, maybe before merging it, could you try to run it from the branch I created in the main repo
s3_index_validation
@Ramon Massoni? To double-check that if fixes the issue
✅ 1
r
yes, I can definitely try! I'll get it done today and let you know. Thank you very much for your kind help Jose, Jonathan and Maxime
👍🏻 1
j
Hi, did you have the opportunity to test it?
Hi @Ramon Massoni Would be great if you could confirm if the fix worked to merge the PR Thanks 😊
r
Hi team, sorry for the delay, I got caught up in another project. I'm checking everything as we speak, will report shortly
ok, pipeline is running
Copy code
nextflow run nf-core/atacseq \
    -r s3_index_validation \
    -resume \
    -profile nygc \
    --input "$SAMPLESHEET" \
    --outdir "$OUTDIR" \
    --genome GRCh38 \
    -c "${PROJECT_DIR}/epigenome_cell_lines/scripts/1-mapping/ATAC/override_igenomes.config" \
    --read_length 50 \
    --narrow_peak \
    --save_reference
will keep you posted
I get this error (+ explanation from Claude)
Copy code
The s3_index_validation branch fails on Nextflow 24.04.2 with a compilation error:


Module compilation error
- file : workflows/atacseq.nf
- cause: Variable `channel` already defined in the process scope @ line 118, column 26.
       def topic_versions = channel.topic("versions")
                            ^
The PR changed Channel (uppercase) to channel (lowercase) throughout <http://prepare_genome.nf|prepare_genome.nf> for "strict mode syntax", but workflows/atacseq.nf still uses channel as a local variable name. Nextflow 24.04.2 interprets lowercase channel as a variable reference, not the Nextflow Channel factory class, causing a name collision at line 118.

Bottom line: The branch can't even launch on Nextflow 24.04.2, so it's untestable — and separately, it still doesn't address the queue-vs-value channel root cause for BWA_MEM receiving 0 tasks
j
I see will revert this and let you know, thanks
r
thank you
j
If you could try again it would be great
I already reverted it
r
submitted! will update you
getting this error
Copy code
[2026-03-30 13:29:42] job START on ne1dc3-015

 N E X T F L O W   ~  version 24.04.2

WARN: Invalid config manifest attribute `contributors`
WARN: It appears you have never run this project before -- Option `-resume` is ignored
Launching `<https://github.com/nf-core/atacseq>` [trusting_wing] DSL2 - revision: e2518260cd [s3_index_validation]

WARN: Nextflow self-contained distribution allows only core plugins -- User config plugins will be ignored: nf-schema@2.5.1
WARN: Invalid config manifest attribute `contributors`
ERROR ~ Module compilation error
- file : /gpfs/commons/home/rmassoni/.nextflow/assets/nf-core/atacseq/./workflows/atacseq.nf
- cause: Variable `channel` already defined in the process scope @ line 118, column 26.
       def topic_versions = channel.topic("versions")
                            ^

1 error


 -- Check script '/gpfs/commons/home/rmassoni/.nextflow/assets/nf-core/atacseq/main.nf' at line: 37 or see '.nextflow.log' file for more details
[2026-03-30 13:30:28] job END
this is the feedback from my AI friend
Copy code
As expected — they only reverted channel back to Channel in <http://prepare_genome.nf|prepare_genome.nf>, but not in workflows/atacseq.nf where the error actually is (line 118: channel.topic("versions")).

And critically — looking at the BWA index section, the revert still shows:


ch_bwa_index = BWA_INDEX ( ch_fasta.map { item -> [ [:], item ] } ).index
No .collect() or .first() — the queue/value channel bug is still there.

Tell them:

Two issues with the latest commit (e251826):

Compilation still fails — workflows/atacseq.nf line 118 still has lowercase channel.topic("versions"). The revert only hit <http://prepare_genome.nf|prepare_genome.nf> but not <http://atacseq.nf|atacseq.nf>.

The original bug is still unfixed — the BWA index path when building from scratch (BWA_INDEX(...).index) is still a queue channel with no .collect() or .first(). Even if the compilation error is resolved, BWA_MEM will still receive 0 tasks.
j
Could you try to run it with a newer nextflow version? e.g. 25.10.4
r
yes, let me try
with
Copy code
module load Nextflow/25.10.0
seems to be running well so far
👍🏻 1
sorry to bother again with this.. I could run the pipeline but, BWA is still not starting..
Copy code
executor >  slurm (45)
[cb/639c94] NFC…GENOME:GTF2BED (genes.gtf) | 1 of 1 ✔
[4c/4040ab] NFC…PREPARE_GENOME:TSS_EXTRACT | 1 of 1 ✔
[c7/7a66b1] NFC…_GETCHROMSIZES (genome.fa) | 1 of 1 ✔
[3f/72c7c0] NFC…_AUTOSOMES (genome.fa.fai) | 1 of 1 ✔
[1b/bc06c7] NFC…_REGIONS (genome.fa.sizes) | 1 of 1 ✔
[49/175f61] NFC…NOME:BWA_INDEX (genome.fa) | 1 of 1 ✔
[63/7e1891] NFC…ECK (atac_samplesheet.csv) | 1 of 1 ✔
[3e/f6c83e] NFC…:FASTQC (U-RT1_WT_REP2_T1) | 18 of 18 ✔
[e8/1a9fe5] NFC…MGALORE (U-RT1_WT_REP3_T1) | 18 of 18 ✔
[-        ] NFC…EQ:FASTQ_ALIGN_BWA:BWA_MEM -
[-        ] NFC…ATS_SAMTOOLS:SAMTOOLS_SORT -
[-        ] NFC…TS_SAMTOOLS:SAMTOOLS_INDEX -
[-        ] NFC…TS_SAMTOOLS:SAMTOOLS_STATS -
[-        ] NFC…SAMTOOLS:SAMTOOLS_FLAGSTAT -
[-        ] NFC…SAMTOOLS:SAMTOOLS_IDXSTATS -
[-        ] NFC…CARD_MERGESAMFILES_LIBRARY -
[-        ] NFC…CARD:PICARD_MARKDUPLICATES -
[-        ] NFC…ATES_PICARD:SAMTOOLS_INDEX -
[-        ] NFC…TS_SAMTOOLS:SAMTOOLS_STATS -
[-        ] NFC…SAMTOOLS:SAMTOOLS_FLAGSTAT -
[cb/c7965f] NFCORE_ATACSEQ:ATACSEQ:IGV     | 1 of 1 ✔
[64/2ae3cd] NFCORE_ATACSEQ:ATACSEQ:MULTIQC | 1 of 1 ✔
Plus 51 more processes waiting for tasks…
-[nf-core/atacseq] Pipeline completed successfully-
Completed at: 31-Mar-2026 22:18:52
Duration    : 1h 9m 1s
CPU hours   : 1.6
Succeeded   : 45
j
umm, I checkout the
s3_index_validation
and run the sample test files and it runs the bwa process, I used this command:
Copy code
NXF_VER=25.10.4 nextflow run <http://main.nf|main.nf> \
	--input <https://raw.githubusercontent.com/nf-core/test-datasets/atacseq/samplesheet/v2.0/samplesheet_test.csv> \
	--outdir ./results \
	--genome GRCh38 \
	--read_length 50 \
	-profile crg \
	-ansi-log false
And here is part of the log:
Copy code
e4/034087] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T150_SE_REP1_T1)
[f1/d9fb6a] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T150_SE_REP1_T2)
[d0/049e0c] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (OSMOTIC_STRESS_T150_SE_REP1_T2)
[b6/b919b0] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (OSMOTIC_STRESS_T15_PE_REP1_T1)
[6a/520a3c] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T15_PE_REP1_T1)
[f6/a66117] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (OSMOTIC_STRESS_T15_PE_REP1_T2)
[9d/704268] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (OSMOTIC_STRESS_T15_PE_REP1_T2)
[f7/a4fbe9] Submitted process > NFCORE_ATACSEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES (genome.fa)
[38/80dedb] Submitted process > NFCORE_ATACSEQ:PREPARE_GENOME:GET_AUTOSOMES (genome.fa.fai)
[72/d4bef0] Submitted process > NFCORE_ATACSEQ:PREPARE_GENOME:GENOME_BLACKLIST_REGIONS (genome.fa.sizes)
Staging foreign file: <s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Sequence/BWAIndex/version0.6.0>
[a0/7d2210] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T0_PE_REP1_T1)
[b1/79e881] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T150_SE_REP1_T2)
[e7/04ed4b] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T150_SE_REP1_T1)
[8b/c66467] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T0_PE_REP2_T1)
[52/bba807] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T100_SE_REP2_T1)
[68/4268fb] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T100_SE_REP1_T1)
[19/7c2a42] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T15_PE_REP1_T1)
[16/70e28f] Submitted process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (OSMOTIC_STRESS_T15_PE_REP1_T2)