TLDR: User Intervention Points for decision-heavy ...
# pipeline-resource-optimization
h
TLDR: User Intervention Points for decision-heavy pipelines? Hi all, Thinking about the possibility of pre-specified "user intervention points" in pipelines with steps that need some user analysis and decision-making. Motivation: Currently, the pipeline designer is faced with 1. choosing defaults that may not be appropriate for a given use-case, 2. automating decisions, or 3. running all permutations in the decision space. Option 1 may cause multiple, wasteful runs of the same pipeline, with tweaked parameters each time or worse, outputs that misrepresent the biology due to mismatch in parameter selection and experiment design. Option 2 is great where feasible, but isn't feasible in many cases. Option 3 is incredibly wasteful of compute resources. So the idea is to implement hard-coded or user-selected break points in the pipeline where all resources are relinquished and the user is notified. User then reviews relevant plots/metrics, selects their choice of parameter to move forward, and resumes the pipeline. Thoughts?