Been talking with <@UEM37TBAR> for module configur...
# team-maintainers
e
Been talking with @James Fellows Yates for module configurations, and we were thinking..what do the maintainers think about starting changing
withName
blocks in configs to use full pipelinesubworkflowsmodule names? pros: 1. mega-pipeline chaining-ready 2. difficult to make mistakes on which config is used for which module when module is used in multiple places 3. local nf-tests should run without problems (i.e. null meta.stuff that are used in prefix) 4. can also make more general with this
^PIPELINE_NAME.*MODULE_NAME$
suggested by @Nicolas Vannieuwkerke cons: 1. porting effort 2. bloated text visuals 3. ?
👍 2
👍🏻 2
🤔 1
n
Being more deterministic is always better IMO. It makes it easier to contribute to the pipeline too
this 2
r
Are we still allowed wild cards to target multiple processes at once?
facepalm literally your last point
😆 3
m
Con: It requires users to understand regex syntax if they want to find what's already been set. Pro: Overriding as a user should be simple as they can copy paste the process name to adjust resources though.
👍 1
Con: Can't use the full name in the test profile config to override the simple name setting in the
modules.config
because modules.config is loaded after the test profiles.
r
Con: Can't use the full name in the test profile config to override the simple name setting in the
modules.config
because modules.config is loaded after the test profiles.
that's a big one
e
What if we provide a nextflow.config in the module's test folder for anything that we want to override?
m
I don't understand. Isn't that what a test profile is essentially?
e
Ok, I misunderstood your point. I see what you mean. I haven't tried changing args or prefix through the test profile. I only set resource stuff, again with the full pipeline name, which does not collide with modules.conf.
s
Con: Can't use the full name in the test profile config to override the simple name setting in the
modules.config
because modules.config is loaded after the test profiles. (edited)
Can we swap the order of loading them?
m
No, because then the params from the test profiles don't get loaded into
modules.config
.
ext.args
that depend on params values wouldn't be set accordingly
😢 1
We would need to use a
-params-file
or cli settings for the test for us to move the test profile afterwards