Also, Júlia wants quick opinions on this: <https:/...
# team-maintainers
j
Also, Júlia wants quick opinions on this: https://github.com/nf-core/tools/pull/3633 1. Should allow
exec
blocks in nf-core modules (I personally don't see why not) 2. If yes, what (if any) specs should we have for inside that block Please put thoughts/opnions in thread below 👇
My thouights: • Allow • Restrictions: for non-memory heavy jobs; still need to do Nextflow/harshil formatting; not allow java-based tool execution tools (i.e., only groovy code)?
r
Seems like there are a few particularities. https://github.com/nextflow-io/nextflow/issues/942#issuecomment-2016957164 ANyone have examples for nf-core/modules? Maybe this is a case were the linting could differ between nf-core and non-nf-core (we had another linting issue recently where nf-core linting is unreasonbly strcit when developing your own nf-core-light modules repo https://github.com/nf-core/tools/issues/3656)
👀 1
j
1. can you summarise the particularities we should keep in mind?
2. I don't think we have any, @Matthieu Muffato is the person who brough the issue up I think also in an extenal modules repo
👍🏻 1
r
no at least not today, I just found that github issue when searching through the nextflow slack about any downsides of using it
👍 1
p
It is pretty summarized by Ben already in there, but • no stdin, env, stdout • files are different (Ben says "tl;dr -- just use val for file inputs and resolve output files against task.workDir" but I wonder about e.g. s3 resources and so on). Some thoughts: • Most important as noted above, nothing heavy (also because resources consumed will be kept around until the end of run). • Agree executions shouldn't be allowed • I think there was something with caching as well (@Mahesh Binzer-Panchal do you recall)? Possibly not suitable for things that can take long to complete either. I can see cases where it makes sense (e.g. for complex logic operations) even though I don't have a good feeling for when it might make more sense to use subworkflows or modules.
❤️ 1
j
Thank you @pontus for the TL;DR, that's exactly what I was looking for. The caching thing might be important...
r
ah yes, I believe
resume
doesn't work
j
That makes me slightly nervous
r
hm maybe that is outdated info, looking at the docs: https://www.nextflow.io/docs/latest/process.html#native-execution we should test this
Copy code
However, it provides additional capabilities such as parallelism, caching, and progress logging.
🤔 1
s
yeah, fairly sure you can use exec with resume nowadays I don't see a use-case for nf-core/modules to be honest though
this 1
j
Because it will always be very custom?
s
I think it would be more for pipeline logic
👍 1
👍🏻 1
m
exec:
processes do cache. As Simon said though, nf-core/modules should usually be running a third party tool.
Ah, this was the issue with exec: https://github.com/nextflow-io/nextflow/issues/6246 or, more specifically,
nextflow clean