Nextflow vs. Snakemake – validation, lineage, and monitoring Hi All, ’m working with a workflow that currently runs in sankemake, and I’m debating whether to migrate to Nextflow or keep building on what I have. Long-term, the pipeline needs to be validated (GMP context), so reproducibility and traceability are key. My main concerns: • Data lineage & provenance – every output must be traceable back to inputs, database snapshots, and parameter sets. • Procedure monitoring – live status tracking and clear audit trails for deviations or reruns. • Updates & maintenance – a clean process for updating tools and databases without breaking validation. • Future growth – deciding whether early migration to Nextflow is worth it Questions for the group: 1. Has anyone migrated a Snakemake workflow to Nextflow primarily for compliance or traceability reasons — was it worth the effort? 2. For regulated environments, does Nextflow provide clear advantages over a well-structured Snakemake setup? 3. How are you handling lineage and monitoring in either framework — any Literature, or best practices you’d recommend?

Check out #C08E444AHSR , they’re doing just that!

Lots to talk about on this thread, but let's start with a few links: • BCO (biocompute object) and RO Crate files for provenance, see https://github.com/nextflow-io/nf-prov • #C05V42SU8DP for a special interest group led by @Alex Peltzer , a sub community within nf-core that's focussed on using Nextflow pipelines within regulatory environments. See also https://nf-co.re/special-interest-groups/regulatory • Tutorial about data lineage, a new native-Nextflow feature for very deep provenance reporting: https://www.nextflow.io/docs/latest/tutorials/data-lineage.html • Seqera Platform, the commercial platform from the developers of Nextflow that builds on top of Nextflow to provide basically all of the features you said you're looking for: https://seqera.io (monitoring, audit logs, we have a lot of customers doing what you're talking about)

I’ll also provide some updates soon on #C05V42SU8DP - we dropped a bit silent there but things are moving forward

I'm one of the main developers of the conversion of the mentioned #C08E444AHSR pipeline. Don't hesitate to ask me any questions on this topic 🙂

Thank you all for the responses! I will look through the provided information 🙂