Has anyone using

DEEPVARIANT_RUNDEEPVARIANT

found that it randomly seems to fail to resume at times? As far as I can tell it’s not something in my pipeline changing, and it happens seemingly randomly so I’ve never had the

dump-hashes

enabled before and after it’s happened unfortunately.

What sort of error did you face ? I ran ~1,150 samples over ~700 species through it. The pipeline runs completed almost without any intervention. I didn’t check if there were actually some job retries (the pipeline is based off the nf-core template, so several LSF codes are captured and retried)

No errors. It just doesn’t resume. I’ve seen it with runs of 1 sample and runs of 40+. Basically I will do

-resume

and about 95% of the time it uses the cached

DEEPVARIANT_RUNDEEPVARIANT

process as expected, but 5% of the time it decides to rerun the process. When it happens it reruns for all samples, so it seems to be something which affects the process as a whole rather than e.g. one sample meta having a bug