Hi @Nils Homer: 1. found data 🙂 maybe check them on nf-core/test-datasets but they now run through the various steps 2. missing modules have now been submitted, but existing ones might benefit of your contribution 3. I’ll be assembly a revision of the new sub-w with the modules on the official repository, based on your pipeline: after the hackathon my time is going to be limited so any contribution you could make would be great. I can give you write access to the fork where I’m going to prep the new code

@Francesco Lescai I really appreciate the effort! My time will also be limited over the next few months unfortunately, so I may not be very responsive on nf-core/modules

Hi, I’ve a question that isn’t directly related to fastquorum. But I would figure that some people on this channel might help me. I’m looking for a decent tool to deduplicate UMI’s and call consensus reads without using mapping coordinates. Any feedback or tool is appreciated!

Hi, I am Anushi Shah, Bioinformatics research officer from Children's Cancer Institute (Sydney, Australia). I am looking to apply fastquorum on our UMI duplex datasets.

Hey folks, I am working on some significant updates to bring

fastquorum

up to date with the new nf-core templates. Stay tuned!

Is there some suitable test data available?

I am starting to gather that too

Can we help you with anything?

<!subteam^S016E36N7TQ> I am looking for two reviewers for our first release of fastquorum 🎉🎉🎉. I would also request that someone from the core-team activate the Zenodo functionality for this repository. I am excited to talk about this first release this week in Boston!

Are you there at the hackathon tomorrow?

I am arriving late Wednesday night, but Jason Fan will be there tomorrow night and Wednesday

There should be a bunch of core team folk at Boston so maybe easiest to follow this in person ☝🏻 But if needed, you can also ask in #CQY2U5QU9 👍🏻 (only one core team reviewer is needed, and we should probably relax that now that we have a <!subteam^S044DU0NG68> )

Yeah, one been asking one community and one maintainers :), best to ask on #CQY2U5QU9 and ask to trade. I think someone else asked for a pipeline review recently

@James Fellows Yates I really love your diagrams. A lot. Than-you! Do you have a favorite tool to edit the SVGs? I tried creating my versions in draw.io, but it wont load the SVGs (in my hands). If you're able to make these changes, I think we're good to go: 1.

fastquorum_subway_alt.svg

a. Can you add the tools to Phase 1 like you did for Phase 2? b. Can you add a space between

fgbio

and

CallMolecularConsensusReads

? In both R&D and HT versions? c. Would you think it's nicer if the text were left-aligned? 2.

fastquorum_subway_desc_alt.svg

a. The right-bottom diagram should be

Phase 2

(not

Phase 3

) b. Would you think it's nicer if the text were left-aligned? c. Any chance you can make "High-throughput" could not be split across lines? d. Can the combined width of the two Phase 2 diagrams be the same as the width of Phase 1? This may help with 2(c). 3. Generally: a. When I open the SVGs, I don't see the names of the file (e.g. FASTQ, BAM). Though I see them on the PNGs you generated. Any ideas?

Megatests

Hi all, very excited about the

1.0.0

release, congrats to all. I am trying to run the multilane sample sheet (

samplesheet.multi_lanes.csv

) that exists in the test-datasets branch but it seems to be failing in the

GroupReadsByUmi

step (I am pretty sure the problem is in the merging though)? Is this smth "known"? I can open an issue or provide more details if not. I did run it in a full genome first, I am now trying with the chr17 that's in the test-datasets.

Thanks for making this tool! I'm trying to set --filter_min_reads in my pipeline. I am getting

--filter_min_reads: expected type: String, found: Integer (1)

but even passing

--filter_min_reads "1"

does not seem to solve this issue. I do understand why it is expecting a string for duplex sequencing, but for non-duplex an int does make sense, and I can't figure out how to set this variable as a string from the command line.

Why do I get "N" as a base calls in my consensus reads after filtering?

Hi all. Thank you for updating the dev version it's working super well for me now. I was wondering what your thoughts are for reads with adapter or polyG content. If I understand correctly, bwa mem will soft clip these sequences? Or would you advise trimming prior to running fastquorum? I'm not sure if trimming may affect consensus calling.

Is there a quality metric associated with the umi / barcode grouping? e.g. "UMI1" is identified to be a set of n reads using some strategy. In the simpliest case of identity for example, all have the same UMI read, but of varying quality. Is there a set of metrics to describe the confidence of the grouping to UMI1? This is a little complicated as it of course depends on not only the DNA sequence quality, but also the UMI length (and if we're really going to model error "correctly" the representation of bases or other UMIs and their edit distances).

@Zachary Norgaard @Simon Pearce I’d like to introduce you too as I’m taking some time off and haven’t had a chance to get to a release.

<!subteam^S016E36N7TQ> we're working on a release of #C0453Q2SFCM and hit a snag. There are 3 checks labeled "required" that are stuck in "Expected". They are labeled "EditorConfig", "Prettier" and "Run pipeline with test data". These are only appearing on PRs to

master

. The bigger problem being I can't seem to track down what's supposed to trigger those checks. With Nils out for several months, I was wondering if you could help us get this resolved either in figuring out what's triggering them or turning them off in the branch protection rules. PR for reference: https://github.com/nf-core/fastquorum/pull/70

<!subteam^S016E36N7TQ> Working on the release with the latest template updates and had a question about the new conda environment schema. It looks like the template update had some conda environment yaml files (specifically samtools ones) update to a new schema which explicitly prohibits a name. This change wasn't made in all the nf-core environment yaml files. On the pull request from dev to master we get failing conda tests and I think it's because of environment clashes.

java.nio.channels.OverlappingFileLockException

https://github.com/nf-core/fastquorum/actions/runs/11707793386/job/32607992632?pr=89 Should I be updating all the environment yaml files to explicitly use the new schema, or should I remove that schema and add "name" back to all the environment yaml files?

Hi, I am trying to run the latest

v1.0.1

of fastquorum with my raw fastq files, but looks like only

fastq.gz

and

fq.gz

(meaning only the compressed files extensions) are supported. Though I can easily change it to fastq.gz, I just wanted to ask in here if there's any particular reason to why the

.fastq

or

.fq

file format are not supported ? Posting the command and the error below

nextflow run nf-core/fastquorum -r 1.0.1 -profile docker --max_memory "63.GB" --max_cpus 16 --input samplesheet.csv --outdir /bp/analysis/127997/results    --genome GRCh38 --mode rd

N E X T F L O W  ~  version 23.04.3
Pulling nf-core/fastquorum ...
 downloaded from <https://github.com/nf-core/fastquorum.git>
Launching `<https://github.com/nf-core/fastquorum>` [fervent_mestorf] DSL2 - revision: 6d63951109 [1.0.1]
Downloading plugin nf-validation@1.1.3


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/fastquorum v1.0.1-g6d63951
------------------------------------------------------
Core Nextflow options
  revision                  : 1.0.1
  runName                   : fervent_mestorf
  containerEngine           : docker
  launchDir                 : /media/raid0/bp/analysis/127997/fastquorum
  workDir                   : /media/raid0/bp/analysis/127997/fastquorum/work
  projectDir                : /home/ec2-user/.nextflow/assets/nf-core/fastquorum
  userName                  : ec2-user
  profile                   : docker
  configFiles               :

Input/output options
  input                     : samplesheet.csv
  outdir                    : /bp/analysis/127997/results

Main options
  duplex_seq                : true

Read grouping options
  groupreadsbyumi_strategy  : Paired
  groupreadsbyumi_edits     : 1

Consensus reads options
  call_min_reads            : 1 1 0
  call_min_baseq            : 10

Consensus filtering options
  filter_min_reads          : 3 1 1
  filter_min_baseq          : 10
  filter_max_base_error_rate: 0.1

Reference genome options
  genome                    : GRCh38
  fasta                     : <s3://ngi-igenomes/igenomes//Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa>
  dict                      : <s3://ngi-igenomes/igenomes//Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.dict>
  fasta_fai                 : <s3://ngi-igenomes/igenomes//Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa.fai>
  bwa                       : <s3://ngi-igenomes/igenomes//Homo_sapiens/NCBI/GRCh38/Sequence/BWAIndex/version0.6.0/>

Max job request options
  max_memory                : 63.GB

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/fastquorum for your analysis please cite:

* The pipeline
  <https://doi.org/10.5281/zenodo.10456900>
  <https://doi.org/10.5281/zenodo.11267672>

* The nf-core framework
  <https://doi.org/10.1038/s41587-020-0439-x>

* Software dependencies
  <https://github.com/nf-core/fastquorum/blob/master/CITATIONS.md>
------------------------------------------------------
ERROR ~ ERROR: Validation of 'input' file failed!

 -- Check '.nextflow.log' file for details
The following errors have been detected:

* -- Entry 1 - fastq_2: FastQ file for reads 2 cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz' (<s3://samplebucket/SRR22795549_2.fastq>)
* -- Entry 1 - fastq_1: FastQ file for reads 1 must be provided, cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz' (<s3://samplebucket/SRR22795549_1.fastq>)

Not strictly fastquorum, but is umi consensus building using fgbio related. I've discovered that there is an nf-core subworkflow called

fastq_create_umi_consensus_fgbio

. This has been broken since last summer when I bumped the nf-core fgbio modules and added the compulsory arguments to the callXconsensusreads modules in https://github.com/nf-core/modules/pull/5624. I've made a PR to fix this subworkflow and swap to nf-test, but I have a question to @Nils Homer and @Zachary Norgaard in particular. Previously, this module was using

SAMTOOLS_VIEW

to filter the bam files with

-f 1

, but I don't think that fastquorum does this. Is this actually required, or can I remove this from the subworkflow (as I currently have)?

// if using duplex UMI paired strategy must be used and therefore
    // only BAM files with all reads paired will be accepted
    // to avoid groupReadsByUmi throwing an error we must filter the zipped BAM

Pipeline release! nf-core/fastquorum v1.2.0 - 1.2.0! Please see the changelog: https://github.com/nf-core/fastquorum/releases/tag/1.2.0

Hi all! I am new to nf-core, so you'll have to forgive any ignorance that I might show. For UMI consensus calling, according to the manual, it looks like this pipeline expects the UMI to be concatenated on the R1 (fwd) or the R2 (rev). My data comes in R1 (fwd), R2 (UMI), and R3 (rev). It doesn't appear that this pipeline is designed to handle data in the format I get. Am I missing something here?

Hi all,

I'm new to nf-core/fastquorum and have a question about the alignment strategy. It seems that all "bwa-mem" commands in the ".command.sh" files come with the "-p" tag. Given that I explicitly run the pipeline with "--duplex-seq false" since I need only single-end reads for my experiment, doesn't that defeat the whole purpose? I do not wnat the aligner to think that my data is pair-ended at all...

One to watch! https://github.com/fulcrumgenomics/fgumi