Hallo there! A small intro about me before I promote my tool. I spent two decades building software for a safety regulated industry, the last ten years on platform and compliance tooling, where every requirement change must show mechanically what was affected, re-verify it, and keep the evidence available. While learning bioinformatics pipelines, I kept tracing the challenges and the equivalent tooling for traceability and impact analysis. A container turns out to be buggy, a sample fails QC after its data was already used, a reference gets bumped. The command -resume fixes the run, but nothing answers the questions that come before and after it, like which of the past runs used the defective input, which other runs consumed their outputs, and which results already left the building. Nextflow's lineage store and nf-prov record what ran, but turning those records into a decision (what to delete, re-run, or disclose now) was something I couldn't establish. So I attempted to build Clew. It reads the lineage data Nextflow already writes (the native lineage store on 25.04+, nf-prov RO-Crates, or plain work/ symlinks on any version), merges runs into one graph, and computes what to delete, re-run, or disclose for a given trigger, for example a donor who withdrew consent, a sample that failed QC, a container with a bug, a reference genome that got updated. It even traces across pipelines, so a withdrawn sample in an rnaseq run reaches the differentialabundance run that consumed its count matrix. Every verdict cites a rule and replays offline, so the output is something you can hand to an auditor. Two minute demo on a real sarek run, stdlib only, AGPL: pip install clew-lineage then clew demo github.com/QuietFlare/clew I am still new to the domain and learning the tooling and process by building in the open, so I am happy to discuss the scope and how it could work for you. Feedback from people who run pipelines for a living is exactly what I need. Issues and PRs very welcome. Thank you in Advance!

Super interesting, because it touches one of the most important things in science: reproducibility and the ability to predict downstream impact. This is close to something I've been working on in horus-runtime. Right now it only fingerprints task inputs to decide whether to skip or re-run a task, so it's more about saving compute and time than actual impact tracing. Yours looks a lot more robust since it traces through the whole workflow graph and actually tells you what to do about a change (delete, re-run, disclose), not just that something changed. Would you be up for a quick chat about your implementation decisions? Curious how you're modeling the cross-pipeline lineage.

Hi @cdominguezdalmases Thanks! Your use case sounds interesting. Happy to discuss with you on my approach. Please DM me and we'll find a time!