Hello everyone! With the new implementation of `t...
# random
j
Hello everyone! With the new implementation of
topic
, should we change all nf-core modules/workflows/subworkflows to topic?
p
I was thinking yes (with
eval
) for all #modules to collect software versions at least 👍🏻
Do you have other broad suggestions?
j
I also think the same! The question is, if we start by the modules, we would break workflows, since they rely on the version, is that correct?
Or do you mean just changing the version output channel, then changing to topic later?
r
Only once a developer would update the module in which case they can then update the code collecting the versions. Or do you mean something else?
j
Yes! So that was my point: How to do it in the best manner
p
Yeah it needs some careful thought. I've been putting it off a bit 😆
@Rike Hanssen the problem is that it's a pipeline level change, so all modules in a pipeline need to make the switch at the same time
I guess this means making a new lint test in #tools that forces pipeline devs to use the new syntax. That will in turn force them to go and update all of the #modules in their pipeline.
The only alternative I see is doing a huge mammoth effort to update all modules. But that's getting to be increasingly unfeasible as there are so many.
Unless we can script the change somehow
r
Maybe we should collect batch changes we need to make at this point….
Ah I thought you could mix and match and in the end mix the topic and not topic channels together. Good to know
p
Well maybe we could 🤔
But I think it'd be nicer to do batch tbh
I think it'll get confusing otherwise
e
I think we need a way to pull in the legacy
version.yml
and then the new
topic
channel
I think most of them are going to require some work
p
Ah good thinking - so then we could wire up the topic channels with the old (pre-eval) module syntax?
Makes sense. Then we can separate the two issues and do them async
e
Exactly
And we're not just caught in limbo
👍🏻 1

https://media.githubusercontent.com/media/edmundmiller/presentations/d0348c4233c3a81061a3ed12b34b294b4c400761/2023-10-20/img/nf-core_swap.jpg▾

💯 2
Just like the nf-test adoption
p
We have a bunch of module syntax changes. It'd be good to try to collect them all somewhere so that we can do them in one pass
(And write lint tests to flag them)
e
My bad, I thought I shared this here too not just office hours https://share.stashpad.com/doc-eGLXAHFgV--CVbJZekhjixyS
The real hero behind it all is tools
Slowly guiding every PR like a river through a canyon
😆 1
p
Perfect 👌🏻
What's "channel thing for conda environments"?
I added a few extra 👍🏻
e
@Florian Wuennemann? Maybe I wrote it?
f
You wrote that @Edmund I haven't added anything yet to the stashpad 😉
p
Bed for me now, but it'd be nice to write this up as a pinned issue in #modules with sub-task issues for each item if possible.
I'll try to find time to do it tomorrow if no-one beats me.
m
Just a heads up for bulk changes: there is a strict matrix limit of 256 tests in a test strategy on github, so that is the max number of module changes in a PR.
e
r
it was me. Linked the slack thread. It was about stripping out the default channel. Accidentally put it in twice. Removed the duplicate
👍🏻 1
p
Just looking at the blog post I wrote the other day, and with the example I had for
eval
and
topics
I'm struggling to see how they can be done separately again: https://nextflow.io/blog/2024/nextflow-2404-highlights.html#collecting-software-versions
Because the topic is defined within the process output:
Copy code
tuple val("${task.process}"), val('fastqc'), eval('fastqc --version'), topic: versions
So it kind of feels like we should do both together 🤔
@Edmund did you have an idea of a syntax that we could use to collect topic channels from modules with the old style version outputs?
Ok, I updated https://github.com/nf-core/modules/issues/5828 to use an issue for each. I wrote a lot of issues and tried to write some background and context for each one.
I'd like to move this discussion out of this thread now, and have it on #modules instead.
e
did you have an idea of a syntax that we could use to collect topic channels from modules with the old style version outputs?
Nah, just a dream.
💭 1
We could write them all to a yaml maybe? I actually looked at this with Steve in Boston. He wants to get rid of all the yaml intermediaries
I'll comment on the issue if I have any ideas
p
He wants to get rid of all the yaml intermediaries
Ben wants the same - I'm pretty sure that there's a POC somewhere where he did it already
👀 1