Hey all, I'm pretty new to using the NF-Core pipe...
# documentation
c
Hey all, I'm pretty new to using the NF-Core pipelines, and first and foremost, I wanted to say that they are phenomenal! I hope that everyone who has worked on them is very proud, as they represent incredible accomplishments! This has probably been suggested before, but I just wanted to mention an (admittedly rather involved) idea I had when it comes to documenting the outputs of the pipelines. Thanks to the user-friendly designs, I've found it easy to set up and run the pipelines, but given their increasing complexity, one often ends up with a ton of outputs that can be somewhat difficult to understand in relation to the overall pipeline. I was thinking that it would be helpful if the "Output" tab of the NF-Core pipeline documentation had a place where you could paste in the command you ran (potentially with the samplesheet too, as needed). Then, ideally, the platform would detect which settings you specified and know the defaults for the ones you didn't. Then, it would output a visualization of the pipeline that actually ran. One would then be able to hover over the steps in the visualization and see which outputs are expected for each. You could also click on those outputs to get detailed information explaining their contents. Here's a mock-up of this for a segment of the Airrflow pipeline. Obviously, creating something like this would be a ton of work, but since the Nextflow team doesn't appear to be daunted by ambitious undertakings, I figured I'd mention it.
Come to think of it, AI will probably be able to do this on the fly in another couple of years, so it's probably not worth spending human time on it in the interim.
j
It's indeed a very nice idea, to have expandable/zoomin-able metro maps - particualrly if you could link it with a particular run. The problem currently simply is that the metromaps are not currently automatable in a useable way ... so it's quite a way off. And making a good metromap still involves quite a lot of design 'involvement' by a human in how to make an effective - they are not one-to-one representations of the entire pipeline - they are very much simlpified for better user-level understanding
👍 1
For example you can compare this metromap for a relatively simple pipeline:
image.png
With it's DAG
So it's a great idea, but will be a long time off
(in my personal opinion )
e
nf-metro plugin!
j
Yeah the nf-metro is one step
But currently still hand-made
There is also the unpublished metroflow tool
but also I don't know if it does the simpliication
c
Thanks for sharing the current state of things James! I figured it had already been thought of and that it would be a ton of manual work
j
Yeah, but you're definitely summarising what aot of people want so we will get there eventually 💪
🙏 2
p
installing nf metro!!!! 🙂
I love that it produces svg so we can tweak the output