A community effort to collect a curated set of analysis pipelines built using Nextflow.

nf-core

So looking at <@U020B09EJSZ> functions in <https://github.com/ITMmalaria/plasmovar/blob/dev/subworkflows/local/utils_nfcore_plasmovar_pipeline/main.nf>, reminded me another forgotten plan.
Basically I wanted to pull in the functions from sarek <https://github.com/nf-core/sarek/blob/dev/workflows/sarek/main.nf#L643-L716> into <#C08TKFDUNHG> as I believe these are the king of things that we can share in between pipelines quite easily and I do believe a plugin would be a good idea for that.
Main question now is do we want a dedicated plugin for these, as I do believe that these functions from Pieter would be a good addition, and something nice to have in sarek, and other similar pipelines, or do we want to pile in <#C08TKFDUNHG>. cc <@U039DEJ1RJS> <@U03CKGEU3LZ> <@UPC8CHDKQ>

There’s some stuff that should go to `nf-schema` IMO, but all the fastq things would be really nice

ok, so what should go where?
do we need a new plugin?
like `nf-fastq-validation`?

Not sure I agree with the readgroup handling though, but thats a matter of opinion. I tend to follow the Illumina defaults

&gt; like `nf-fastq-validation`?
Not sure if “validation” is the right word here :thinking_face:

I'd prefer file specific functions to be in a plugin called `nf-&lt;file_type&gt;` so `nf-fastq` for example in this case

All functions that don't have an obvious place could go in <#C08TKFDUNHG>

Although that all file processing should a process IMO but that's another discussion

&gt; I tend to follow the Illumina defaults
for reference, this is how we do it
<https://github.com/nf-cmgg/preprocessing/blob/main/subworkflows/local/utils_nfcmgg_preprocessing_pipeline/main.nf>

&gt; Although that all file processing should a process IMO but that’s another discussion
This is not file processing per-se; more like a range query to get the first line and parse it, so no heavy transfers for the head job

I'll have a more indepth look later, because I was trying to follow the illumina defaults too :sweat_smile:

Not a <#C0749PAEU9F>, no worries :wink: sounds good in this case indeed

<@U020B09EJSZ>, we mimic the RG field extraction from `bcl-convert`

fwiw, if you use `bcl-convert` you get almost all of the rg processing for free

You can add custom fields to the samplesheet for extra RG fields and you get a nicely formatted tsv with RG fields for each sample

Btw: I remember I had to make a bunch of exceptions for SRA files because their headers are sometimes different. Just a thing to keep in mind here too.

yeah, you can do bclconvert if you do the sequencing, but if you get fastq files from somewhere else...

yeah, I know, that’s why the fastq functions mimic the bcl-convert behaviour, so there’s less mismatches