I could use some help from the config magicians for our institutional pipelines 🧙🏻 We’ve got a series of workflows over at nf/cmgg, with our own version of #CGRTMASKY. Common plugins for our infra are defined in the configs repo. Whenever we activate the

nomad

profile, all other plugin imports for the configs repo are overwritten. We’d like the have them all. Please send help, you’re my only hope

I don't know if it helps, but I seem to remember that sub

profiles

can have a different behaviour than straight

-c

configs...

only the nomad plugin, since it’s the only one that has an extra

plugin

block

I think you might be stuck between a rock and a hard place... to me it sounds like a problem with

nomad

or

nextflow

...

I know we used to be unable to “merge” plugin blocks, but that should’ve been fixed in NXF_VER>26 according to Ben

I don't follow, where's the second layer when not running nomad?

@Matthias De Smet I think I was mistaken when I originally told you about this. It turns out that the v1 config parser never supported merging plugin blocks. the v2 parser accidentally supported it in some cases but not others: https://github.com/nextflow-io/nextflow/issues/6754 so it is actually a grey area at the moment. I need to review the implications of explicitly supporting it and (hopefully) resolve the grey area by 26.10 e.g. merging is convenient in this case with nomad, but overwriting gives the user full control over which plugins are loaded. on the other hand, removing a plugin declared by the pipeline (like

nf-schema

) would just break the pipeline

🙀 Just to check my understanding, the v2 parser don't have any hijinks like where a

{..}

block could redefine things whereas the same would work if written

a.b = c

?

no, that was a quirk of the v1 parser

🤔 would you be able to document the exact behaviour somewhere?

that's the thing, there is nothing to document because currently the behavior is undefined 😅 that is, any behavior is accidental

Yeah, I mean, can we document the “accidents” somewhere and make it a formal spec/requirement for the robot overlords/dev minions to take into account?

I don't know how others see it, but generally documenting accidents like this make it much harder to fix them to behave sanely. For now I'd recommend making sure assuming the worst of the world. I haven't tried now but have the feeling you should be able to build up a representation of your needed plugins in

params

and deduce a filename to include that gives a suitable

plugins

clause at the end.

Could document here in the meantime: https://midnighter.github.io/nextflow-gotchas/

@Ben Sherman (or anyone else) is there any version of the code below that could work?

plugins workflow.profile.contains('nomad') ? {
    id 'nf-cgroup-metrics@1.0.1'
    id 'nf-cmgg@0.2.5'
    id 'nf-teams@0.1.1'
    id 'nf-nomad@0.4.0'
} : {
    id 'nf-cgroup-metrics@1.0.1'
    id 'nf-cmgg@0.2.5'
    id 'nf-teams@0.1.1'
}