In the <#CE8SSJV3N> pipeline, the PICARD_MARKDUPLI...
# nostupidquestions
v
In the #CE8SSJV3N pipeline, the PICARD_MARKDUPLICATES process takes a long time and blocks most many of the downstream processes. It also does not scale across processes well. Is there a way I can prioritise this process above salmon_quant, for example? I think it would speed up overall run times.
m
Depends on your execution environment. Are you using Slurm? If you have a priority partition there, you could submit those tasks to a different partition for modify the niceness. But ultimately duplicate marking is simply a slow process. You could skip it, if you are not interested in an estimate of the duplication rate, since truly authoritative marking and removal would require using UMIs anyway. New tools are being developed, but have not yet been released and added to the pipeline. Maybe that new FPGA-supported tool will have a pure CPU mode that is faster than picard?
v
This is a local execution. Yes, a faster tool would be nice.