A draft of the new guidelines for modules: <hackmd...
# wg-meta-pipelines
n
A draft of the new guidelines for modules: hackmd.io/YmVPL55sTomtFIpNKlRKBA I'm still working on it but feel free to comment if I did something stupid
👀 3
j
I would argue strongly that
prefix
should be a required input as well as
args
. Otherwise subworkflows still have to depend on config
n
Yeah we're currently also discussing this 😛
❤️ 1
j
Otherwise this looks really nice <3
m
The one main contention we have is for example chaining modules e.g. samtools sort -> samtools view and if say
samtools_args
is set, we don't want that propagating to samtools view when it's set for samtools sort
n
I'd say we use
samtools_sort_args
and
samtools_view_args
in that case
m
I like that better
j
Either that, or arguments as a list, if we want to maintain the existing pattern better?
Copy code
record(
   ...
   arguments: List<String?>
)

...

tool \\
${arguments[0]} |
tool2 ${arguments[1]}
m
I'm against indexed args. It's very easy to get stuff mixed up
party this 2
n
Yeah I'd like it to be more descriptive
j
Generally agreed, but there are cases it would be necessary: github.com/nf-core/modules/pull/11550
n
That's definitely not a standard module 😅
😇 1
m
but there it would still be more readable to have the tool name in the args
💯 1
j
Though perhaps in the case that there are multiple tools,
arguments
itself should be a record with keys? That would be easier for linting
Copy code
record(
    ...
    arguments: Record
)
n
And we shouldn't make anything else less clear just for this
j
but there it would still be more readable to have the tool name in the args
Yes, until you run samtools view twice 🥹
n
samtools_view1_args
and
samtools_view2_args
?
party this 2
m
We need a method to deal with position placement anyway. There are tools that require global and tool specific options for example
👍 1
j
I'd say we use samtools_sort_args and samtools_view_args in that case
Re this, maybe you could specify the naming SHOULD(?) match the same rules for naming of nf-core modules/subworkflows etc?
m
We're thinking of multi-tool commands here too like those with bwa and samtools
👍 1
j
And then only add numbers when you have repeated use of the same command twice in one pipe (like
samtools_view
)
💯 2
Yes that's what I mean - I don't mean use the module name itself but rather use the same strategy
💯 1
Names in the input SHOULD be descriptive of their function in the process
What does that mean exactly, I think you will need to explain that a bit better
n
Yeah that one still needs some work, I might remove it in general
j
Copy code
record MyProcessArgs {
    tool1_args: String
    tool2_args: String
}

process MYPROCESS {
  record(
      ...
      arguments: MyProcessArgs
  )
But what about putting these all in a record rather than as flat record inputs?
m
I think it will generate more adapter code with little benefit. In the end we just need a few strings inputs
👍 2
Are we missing anything major in the migration example? Do we want more minor things like what happens with
stageAs
,
arity
, a more specific
ext.args + ext.args2 + ext.args3
example, positional arguments?
j
I think a stageAs example would be good (plus perhaps demonstrating how it can be used for "module binaries" by staging a script?).
m
Do you have a module in mind? I thought module binaries were wave only so we never used them.
n
I'm not sure module binaries are possible using stageAs, at least not in a portable way
j
Basically the suggestion from Ben was you could put a script in a
moduleDir
and stage it into the work directory using stageAs: https://nfcore.slack.com/archives/C04QR0T3G3H/p1773668639446329?thread_ts=1769678574.070939&amp;cid=C04QR0T3G3H
This would also mean we could drop template modules.
n
oooh I like this!
Has anyone tried this yet?
m
Need to confirm that works since that was back in march right?
So template's -> module/bin/script.ext ?
b
do it eyebrows
🤣 1