Hey all, quick question.
When you upgrade a varian...
# nostupidquestionsu
user
02/11/2026, 9:24 PMHey all, quick question.
When you upgrade a variant caller version or switch reference/annotation builds, how do you validate that the calls are “stable enough”?
Do you formally quantify drift between versions, or mostly rely on regression tests + spot checks?
p
Phil Ewels
02/12/2026, 6:47 AMThere's no one answer to this really, so the short answer is that validation is up to the end user
Phil Ewels
02/12/2026, 6:47 AMDifferent people will have different validation requirements and processes, so it's impossible to cater to everyone on the nf-core side
Phil Ewels
02/12/2026, 6:48 AMThe best that we can do is have testing in place where possible, and then trust the pipeline devs / community of users that the changes are good ones.
Phil Ewels
02/12/2026, 6:48 AMAnd make it clear in release changelogs about what has happened, so that downstream users are clear
a
Ammar Aziz
02/13/2026, 7:19 AMI suggest having a truthy dataset that you test using every time a verification is required. I say 'truthy' because it's sometimes really tricky to create such a dataset. It could be 15 SNPs that are always present in a population, or a public dataset that is used.
Ammar Aziz
02/13/2026, 7:21 AMI've written for our lab a detailed bioinformatics semver that defines what happens when different parts of the workflow are upgraded. eg patches (1.2.X) don't trigger anything but major version change (X.1.0) triggers a verification.
Ammar Aziz
02/13/2026, 7:22 AMOf course in the SOP you define what pass means.
Ammar Aziz
02/13/2026, 7:25 AMSome people have published benchmarking datasets: https://lskatz.github.io/posts/2023/04/26/datasets.html and if you do create one, then releasing it for the community is super helpful.