Hello everyone, my name is Mert. I am trying to re...
# eager
a
Hello everyone, my name is Mert. I am trying to reproduce the results of papers that use the EAGER pipeline. However, I haven't been able to find any papers on Google Scholar that clearly show their intermediate results. If you know of any such papers, could you please help me out? I would ideally need a paper that shares a complete MultiQC report (perhaps in the supplementary data), or at least includes specific module outputs as figures, such as a mapDamage plot. Thanks!
j
What do you exactly mean by intermediate results?
Can you describe what exactly you're trying to do? That would help guide us - resource usage? just trying to learn the pipeline and want to replicate the analysis? Does it have to be a paper? Unfortunatley it's rare people publish stuff like MultiQC results as they take key metrics and dump it in their own tables because jouranals only want a excel table...
a
I want to validate the EAGER pipeline I created in the Galaxy framework. You can take a look at this link: usegalaxy.eu/u/mert1907/w/ancient-dna-analysis-pese To do this, I want to use a published aDNA paper with public data. I need to run their raw data through the workflow and see if I can reproduce their findings to ensure everything works properly.
@James Fellows Yates You could say that you can run and compare any data using nf-core and Galaxy. However, I want the nf-core side to be based on published results.
👍 1
j
oh cool!
And nice idea :)
I'm on my phone ATM so can't check exactly what I put in there, but have you looked at the Data S1 of the nf-core/eager paper? https://peerj.com/articles/10947/
You could also maybe(?) use the Megatest example results we normally generate on each release: https://nf-co.re/eager/2.4.5/results/eager/results-42c9d5f8602e5e88fdcec28f194d2cd4cff61c75/
Unfortunately eager v2 is very old now in nextflow terms (eager V3 which is in the latest version of nextflow is coming quite soon, see the
dev
bench), so the automated tests haven't been able to be executed for the last few patch releases but not much has changed in them
💯 1
a
I would also try to use the automatically generated outputs if possible, thats the easiest approach usually - you could also run the
dev
version with the test_full profile(s) and use that. The utilized data is coming via the official papers (mentioned in the paper James mentioned above).
a
Hi again. I managed to get identical results for the cod fish dataset. Thank you so much for your help!
For my next step, I am planning to replicate the Barquera (2020) paper. Do you have the execution report for the run where you replicated it? I need it to correctly set up my parameters. The paper mentions min length = 30 and min base quality = 20 for AdapterRemoval, and a default seed length of 32 for BWA. Are these the only parameters I need to change, or is there anything else I should be aware of?
j
@aydinalimert I believe if you go into the supplenment of the nf-core/eager paper
There should be a mardkown document that gives the full command used for the Barquera demonstration 🙂
a
Thanks, I missed the .md file. I have a question about the merging steps. In the cod fish run, there are two samples: cod076 and cod092. Each sample has 3 paired-end reads so 6 fastq files in total. I convert r1 and r2 into a single paired end list in Galaxy. So, my cod076 collection has 3 list:paired inputs. I believe eager does the same using the L1, L6, and L8 suffixes. The 3 paired end (6 fastq files) datasets go into AdapterRemoval. They become collapsed into single files. Then lanemerge concatenates these 3 files. The cod076 collection becomes one single file. This single file is used as input for BWA. However in the Barquera reproduction, it looks like lanemerge is not used before BWA. Damageprofiler outputs show all 4 fastqs for SJN001 separately. SJN001 reads are not merged before mapping unlike the cod fish data. I am confused about how to structure these merging steps properly. Could you please explain?
j
This is because for the Barquera reproduction, the 4 FASTQs are across two independent libraries of the same sample. Therefore these have unique molecules and/or may have been treated differently so it would not be correct to calculate the damage all together (as the 'presentation' of damage is highly dependent on the collection of molecules and the particular chemistry in teh library master mix). In the case of the Fish, they are the same libraries just sequenced across different lanes
a
In this case, merging the lanes belonging to the same library for the same sample at the start of the workflow,before adapter removal,is a right approach, isn't it?
j
You can do it yes. However most people prefer to merge after mapping (but before duplicate removal!)
Because you can then distribute your mapping across a lot of smaller and shorter tasks on HPCs
(eager 3, the one in
dev
does this)
Otherwise if you have deep sequenced data across multiple lanes you have one massive mapping job which hits lots of walltimes/gets hit by resource request peanlisties
a
So, eager 3 performs a bam merge operation between
samtools view
and
markduplicates
?
j
Yeah
exactly
👍 1
(@Thiseas C. Lamnidis should know exactly where)
a
But still, I think there is a catch here. In the Barquera reproduction, the libraries were seperated in the Damageprofiler, but they were merged in the Qualimap results. therefore, the bam merge must have taken place after the duplicate removal.
j
Sorry yes that is correct: you want to damage profile on each library separately (e.g. if they are different strandedness or different polymerases, ), so it would be after markduplicates and before qualimap/genotyping you can see the merge points here: https://nf-co.re/eager/2.5.3/docs/usage/#tsv-input-method; the 'merge UDG treatments' is equivalent to 'merge libraries'
👍 1