https://nf-co.re logo
Join Slack
Powered by
I propose to use topics for reports that are fed t...
# team-maintainers
m
I propose to use topics for reports that are fed to multiqc to simplify publishing via workflow output and passing reports down the line to multiqc. Yeah or nay?
e
Yep, but like versions at the pipeline level, we should have a period where both are acceptable (regular channel and/or topics), right? And when we have transitioned to only topics, we could connect this to what you'd said before, about using MultiQC in the main.nf (instead of the main workflow.nf), to do it once in the future if pipeline chaining becomes a thing
n
IMO we should allow both for reports since reports are sometimes used to define channel logic. But I'm all for the idea!
this 2
m
my gut feeling is the topic name should be 
qc_reports
or similar. 
reports
feels too generic
👍 1
e
or straight 
multiqc
?
Many tools have various reports and qc tools might have their own
m
there are pipelines that use other tools for qc reporting, so i was hesitant on adding that specific tool, but am not sure tbh
j
I would think any report channel would also have to have the same kind of structure as the versions topic channels:
Copy code
tuple val(PROCESS_NAME), val(tool), path(report)
So that a pipeline can potentially do a bunch of filtering on the report channel to remove unwanted reports (or reports from the same tool from the wrong process instance) prior to feeding into e.g. multiqc
j
I like it. My subworkflows etc tend to have special multiqc channels and this would save me that.
m
another proposal for the topic is multiqc_reports
this 1
let's vote: 1️⃣ reports 2️⃣ qc_reports 3️⃣ multiqc 4️⃣ multiqc_reports 5️⃣ multiqc_files
4️⃣ 3
3️⃣ 1
2️⃣ 1
1️⃣ 1
5️⃣ 10
e
multiqc_files, to keep compatibility with old channel name, maaaybe?
Otherwise, multiqc_reports, if it has to be one from the list
j
Oh I like 
multiqc_files
even better
j
Agreed. I think 
reports
is overloaded wrt mulitqc
m
@Phil Ewels happy with multiqc_files?
👍🏻 1
r
shall we cross-post to #C04QR0T3G3H as they'll be the ones affected?
m
@Nicolas Vannieuwkerke, I updated this PR, and I think it's ok to merge now: https://github.com/nf-core/modules/pull/9677
@Jim Downie thanks for the suggestion, I like it
@Nicolas Vannieuwkerke Jim is suggesting adding task.process
n
It doesn't really hurt to add it, I think 😛
m
Done
I'm thinking that we might update all these syntax with the record types anyway, so I'm fine with this
And since I'm the first one exploring this land, I'm free to do as I see fit
maxime 1
u
Sarek is using it
e
wrong one xD
m
sarek will be using it 😆
😆 3
people complained I play too much with sarek and or rnaseq, so now I play with references and rnavar, and then put things in other pipelines after
It looks amazing
We could probably be even smarter later on with the record types
What do you think people, shall I make PRs to the modules repo?
j
Seems sensible- we're emitting the files to channels as well as sending them to the multiqc topic. I think you should PR one module and let folks argue about the approach for a bit.
m
That's what I wanted to hear
j
My only thought is that we're baking in a multiqc assumption that might not be applicable for modules used outside nf-core, but I don't know how much folks will care. Let's just duke it out on a PR
✅ 1
s
Speaking as someone who uses modules in a non nf-core setting, I just delete bits that I don't want (and then patch to keep them in sync)
m
ok, it has became more of time drain than planned
But I caught a typo in a subworkflow, so not everything was a lost of time, and I have a nice idea for a new function to improve capture filenames or just files actually for nft-utils that would be useful in the modules repo
cf https://github.com/nf-core/modules/pull/9964 @Jonathan Manning I tried to be the least disruptive I could in the subworkflows you made I would actually have loved to strip out more files and just use topics for it, but I do think it's weird that subworkflows are not emitting some files. But in my opinion these emits are only kept for testing purpose, and we should tackle that at the nf-test level, but that's an issue for another day
j
Left a note!
I do think it's weird that subworkflows are not emitting some files
They will have to soon, for the new workflow output syntax
👀 1
m
I agree, but I think it's a current limitation of nf-test that we cannot verify topics
oh, so not a typo but intentional facepalm
commented back
s
We really need to get nf-test to be able to test topics...
💯 1
m
1 step at a time
first strict syntax
n
Soon 👀
🚀 1
2 Views
Open in Slack
PreviousNext
Powered by