Trying to work around the new config issues where you cannot have a function and so I put my code as a closure in a parameter for access later and unfortunately this seems to break

dumpParametersToJSON

which I think passes a

null

item to the closure when it’s trying to serialise this. Any thoughts on how to handle this? This also seems to break schema validation even though the param is in

defaultIgnoreParams

cc @Nicolas Vannieuwkerke

Yes we also noticed that during the migration of nf-core configs to strict syntax. At that point we decided that we should not support parameter types that are not officially supported by Nextflow so I didn't fix this in nf-schema.

Hmm. I was under the impression that setting a parameter to be a closure was now the only way to essentially use functions in config?

Yeah that's true, we however managed to migrate our configs to not use functions anymore. Is this also possible for you?

Not sure. My use case is I set the prefix based on some fields in the meta for various processes (via

ext.prefix

) and I don’t really want to have to duplicate the logic for that across every single process in

modules.config

. e.g. currently looks something like:

ext.prefix = { "${calculate_from_meta(meta)}.somecustompart.someotherpart" }

And that

calculate_from_meta

is going to be dynamic for any given item in the channel and will change as a channel moves through the pipeline and accumulates extra info in it’s meta (so I can’t just compute it once at pipeline start)

This is a case that could be solved my moving this logic to the pipeline itself?

But thenI have to call that function in a whole bunch of channel logic, and if you forget in one place the value it sets can become stale. And this happens across ~20 subworkflows, instead of just modifying the

ext.prefix

in single file. And of course I can’t call it within remote subworkflows without patching. If the

ext.prefix

closure could somehow access a function defined somewhere else in the pipeline this might be able to work?

maybe it can access a function written in the lib folder? The code in the closure gets executed in the process block

Ah that might be the best short-term workaround.

(also if you need to recalculate the same value everytime, couldn't you just make that the ID or some other meta field?)

Aha yes maybe that is the way, but of course then I suspect joining on the meta map/resumability might go haywire

You also got a point there, it's an interesting usecase though