A community effort to collect a curated set of analysis pipelines built using Nextflow.

nf-core

Does anyone remember why we use the default options of `--split-files --include-technical` for fasterq-dump? Perhaps <@UEB97FBN3> or <@U02AYM7HVR7>? Looking at the documentation, the default <https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump#1-fastqfasta--split-3|split-3> seems to me what most people would want, rather than what <https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump#3-fastqfasta-split-file|split-files and include-technical> appears to do.

No sorry, I ported it over from the previous iteration and kept the settings. <@UEB97FBN3> is on holiday for the next 2 weeks. Can we go digging through the Git history?

That might be the least useful diagram I have every seen :laughing:

I think `split-3` will merge overlapping paired end reads, which will cause issues for downstream aligners. `split-files` will create the corresponding paired end read with the sequence content of both in there which is more like what people expect

So `--split-files --include-technical` is the expected input for most aligners, which would make sense for the next pipeline.

Okay, maybe I got it wrong, but having potentially 5 FASTQ output files with different index positions for reads and technical reads seemed wrong.

It always did the right thing for me in practice, mind you, but from the diagrams I would expect something else entirely.

It would make more sense for it to write R1, R2 and R-merged but that would require _every_ pipeline to handle that use case. This is a good use case for storing sequencing data as an unaligned BAM.

The main reason for including both `--split-files --include-technical` this is that it is required for 10X reads to be output correctly. For e.g. 10X GEX files, read 1 contains the cell barcode and UMI sequence, but the read is labelled as ‘technical’. If not specified explicitly, this all-important read had been missing up until it was included). I have never ever seen more than 4 reads in practice, it is mostly limited to 2 files for paired-end files, 3 for 10X GEX, and 4 for 10X ATAC.)

I think the ENA never serves out technical reads at all, so download methods ftp and aspera both do not yield usable data for 10X (or at least last time we checked (we also opened tickets with the ENA back in 2018 or so…))

looking at the documentation of `--split-3`, it looks like it is an ENA-type extraction (without the technical portion of reads), suggesting that it would be useless for single-cell applications (which a lot of data is these days…)