https://nf-co.re logo
Join Slack
Powered by
Hi guys. The test-profile of bamtofastq v2.2.1 do...
# bamtofastq
a
Hi guys. The test-profile of bamtofastq v2.2.1 doesn’t seem to work. I run
Copy code
nextflow run nf-core/bamtofastq -profile test,singularity --outdir out
and it fails with
Copy code
ERROR ~ Error executing process > 'NFCORE_BAMTOFASTQ:BAMTOFASTQ:PREPARE_INDICES:SAMTOOLS_FAIDX ([])'

Caused by:
  Process `NFCORE_BAMTOFASTQ:BAMTOFASTQ:PREPARE_INDICES:SAMTOOLS_FAIDX ([])` terminated with an error exit status (1)


Command executed:

  samtools \
      faidx \
       \


  cut -f 1,2 .fai > .sizes

Command exit status:
  1

Command output:
  Usage: samtools faidx <file.fa|file.fa.gz> [<reg> [...]]
  Option:
    -o, --output FILE        Write FASTA to file.
    -n, --length INT         Length of FASTA sequence line. [60]
    -c, --continue           Continue after trying to retrieve missing
...
A few other jobs did go through, but 
PREPARE_INDICES:SAMTOOLS_FAIDX
fails
Copy code
[-        ] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PREPARE_INDICES:SAMTOOLS_INDEX                  -
[a3/982197] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PREPARE_INDICES:SAMTOOLS_FAIDX ([])             [  0%] 0 of 1 ✘
[83/d74756] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_IDXSTATS (test)      [100%] 2 of 2 ✔
[e3/5baaac] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_FLAGSTAT (test)      [100%] 2 of 2 ✔
[-        ] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_STATS                -
[b0/8a3e8a] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:FASTQC_PRE_CONVERSION (test)  [100%] 2 of 2 ✔
m
screams
a
Just for comparison, I can run the test-profile of v2.2.0:
Copy code
[appsadmin@sw-build-01 test_bamtofastq]$ export NXF_SINGULARITY_CACHEDIR=/ngc/tools/ngctools/singularity
[appsadmin@sw-build-01 test_bamtofastq]$ nextflow run nf-core/bamtofastq -r 2.2.0 -profile test,singularity --outdir out
m
ok, I'll investigate
a
Thanks, Maxime. I’m also trying to investigate a little bit. It seems like the input here is empty.
Not to worry 🙂
f
seems like something is off because if you look at the command there is no file for samtools
its not even rendered in the sh
a
right
I should say that I’m running this behind a pretty strict firewall.
Are you guys able to run the test-profile of v2.2.1 as-is?
f
I mean I updated all snapshots before the release so I suppose yes
a
Not sure I understand. You mean snapshots for nf-tests? As far as I know, the test-profile doesn’t have anything to do with the snapshots for the nf-tests
f
But if I am able to update the snapshots that means that the test profile runs successfully? At least that is my gut feeling. But I will try and run it now.
Because all that nf-test does is run nextflow in the background and I always supply -profile test when I update snapshots
But I run into the same error as you when I just run the test profile so it is not your firewall
👍 1
a
Thanks for verifying that
f
I guess the culprit is here somewhere:
Copy code
// INDEX FASTA
    fasta_fai
        .branch { meta, _fasta_file, _fai_file ->
            is_fasta_indexed: meta.index == true
            to_index:         meta.index == false
        }
        .set { fasta_branch }

    SAMTOOLS_FAIDX(fasta_branch.to_index.map { m, f, _i -> [m, f, []] }, [[],[]])
Because this is related to the fasta
a
yes
around there
f
but why does that work for the nf-tests
because there SAMTOOLS_FAIDX runs
Copy code
nextflow run nf-core/bamtofastq -r 2.2.1 -profile test,singularity --outdir results --fasta <https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.fasta>
that works so its something with the fasta file not being there when you run just the test profile
lets just add the fasta file to the test.config and then we should be good
a
Yes, I was expecting something like that to work.
I’d like to understand a bit more what is going on here
m
test config specify genome to null
so we have no fasta
a
right
m
but we included the fasta in the nf-test test
f
we always add the fasta in each nf-test so I think we should rather have it in the test profile
m
I totally agree with your assessment
a
I wonder how it worked in v2.2.0
Was it fetching the location of the fasta from inside the input-bam-files? (That might be a totally silly question)
I had a look at test.paired_end.sorted.bam. And I didn’t see any reference to a fasta/fa
m
good question, we did not have the fasta in the nf-test tests as well
a
The test-profile is now also working for me in v2.2.1 when I define 
fasta
in 
test.config
as you have done in PR#149.
🎉 1
I did a bit more research, and - as far as I can gather - the transformation of bam to fastq doesn’t require a fasta-file. If that is indeed the case, wouldn’t it then make sense that bamtofastq can also work without a fasta-file?
f
yes technically its needed for the crams
a
Yes, it is needed for cram-input, but not for bam-input, right?
AI gave me the following suggested change for 
subworkflows/local/prepare_indices/main.nf
,
Copy code
// INDEX FASTA only when a real fasta was supplied
    fasta_fai
        .filter { meta, fasta, fai -> fasta && fasta != [] }
        .branch { meta, fasta, fai ->
            is_fasta_indexed: meta.index == true
            to_index:         meta.index == false
        }
        .set { fasta_branch }

    SAMTOOLS_FAIDX(
        fasta_branch.to_index.map { meta, fasta, fai -> [meta, fasta, fai ?: []] },
        true
    )

    fasta_newly_indexed = fasta_branch.to_index
        .join(SAMTOOLS_FAIDX.out.fai)
        .map { meta, fasta, old_fai, new_fai -> [meta, fasta, new_fai] }

    ch_fasta_fai = fasta_branch.is_fasta_indexed
        .mix(fasta_newly_indexed)
        .first()
        .ifEmpty([[:], [], []])
which seems to work
This is what I get with the abovementioned change to 
subworkflows/local/prepare_indices/main.nf
:
Copy code
[67/b678e2] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_IDXSTATS (test2)          [100%] 2 of 2 ✔
[28/0923c5] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_FLAGSTAT (test2)          [100%] 2 of 2 ✔
[84/5e7c9b] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_STATS (test2)             [100%] 2 of 2 ✔
[79/176f05] NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:FASTQC_PRE_CONVERSION (test2)      [100%] 2 of 2 ✔
[d0/12cc9b] NFCORE_BAMTOFASTQ:BAMTOFASTQ:CHECKPAIREDEND (test2)                               [100%] 2 of 2 ✔
[-        ] NFCORE_BAMTOFASTQ:BAMTOFASTQ:SAMTOOLS_COLLATEFASTQ_SINGLE_END                     -
[fb/a126f1] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:SAMTOOLS_VIEW_MAP_MAP (test)      [100%] 2 of 2 ✔
[55/aebe7b] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:SAMTOOLS_VIEW_UNMAP_UNMAP (test2) [100%] 2 of 2 ✔
[62/386d56] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:SAMTOOLS_VIEW_UNMAP_MAP (test)    [100%] 2 of 2 ✔
[da/40aa66] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:SAMTOOLS_VIEW_MAP_UNMAP (test2)   [100%] 2 of 2 ✔
[fa/9bd4d1] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:SAMTOOLS_MERGE_UNMAP (test)       [100%] 2 of 2 ✔
[65/fd1bb3] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:COLLATE_FASTQ_UNMAP (test2)       [100%] 2 of 2 ✔
[73/b31b25] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:COLLATE_FASTQ_MAP (test2)         [100%] 2 of 2 ✔
[b7/cadf1e] NFCORE_BAMTOFASTQ:BAMTOFASTQ:ALIGNMENT_TO_FASTQ:CAT_FASTQ (test2)                 [100%] 2 of 2 ✔
[16/257c90] NFCORE_BAMTOFASTQ:BAMTOFASTQ:FASTQC_POST_CONVERSION (test2)                       [100%] 2 of 2 ✔
[a1/29dd37] NFCORE_BAMTOFASTQ:BAMTOFASTQ:FASTQUTILS_INFO (test)                               [100%] 2 of 2 ✔
[60/9e4ada] NFCORE_BAMTOFASTQ:BAMTOFASTQ:MULTIQC (bamtofastq)                                 [100%] 1 of 1 ✔
-[nf-core/bamtofastq] Pipeline completed successfully-
Completed at: 09-Jun-2026 18:50:54
Duration    : 16m 2s
CPU hours   : (a few seconds)
Succeeded   : 31
And in 
results/reads
:
Copy code
$ stat -c '%s %n' *
110103 test_1.merged.fastq.gz
113138 test2_1.merged.fastq.gz
122696 test2_2.merged.fastq.gz
109186 test_2.merged.fastq.gz
f
I would rather have this checked by validation. So if we have cram files then I want to catch if there is not fasta supplied. And if we have bam files then we can just skip this process entirely.
👍 1
party this 1
a
Sounds very reasonable to me
2 Views
Open in Slack
PreviousNext
Powered by