Hi all 👋 Do you have an example of how, in nf-core, a module can validate which extra parameters are allowed for that specific module? I have a tool that is reused in multiple modules, but the tool receives different parameters depending on the module in which it is used. I would like to avoid users passing

extra.params

that are not valid for a given module. Is there a recommended pattern in nf-core to: • Restrict allowed extra parameters per module? • Validate them at runtime? • Or enforce this through schema validation? Any examples or best practices would be greatly appreciated.

Do you mean restrict what comes in via

ext.args

or just checking some inputs for validity?

For now checking kind of a blacklist of what should not be provided; for example the tool has a parameter

--temp

which overrides the way we handle work folder; I want to blcok that; that the user can't provide that in that step; other cases are less agreessive and could be provided but we don't know what will be the effect on the module. In fact users are now passing them already.

My understanding of

ext.args

is that this is understood to be a way to potentially do insane stuff inside modules and correct usage is up to the person implementing that module in a pipeline

I mean sorry ``ext.arg`` but even if the user can do crazy things I want to block things that do not make any sense.

you can of course try to do some pattern matching on

ext.args

and raise errors, I have not really seen that being done, and I am not sure if it is your job to do that. To me this is maybe something that should be documented.

Does anyone has this problem; I was planning to put a validator; with some parameters that can't be provided; they can play with any other thing but not that one.

you can definitely catch

--temp

and throw an error for that. But we generally say "caveat emptor" - if you are passing things to

ext.args

it is your responsibility for them to make sense, not the module code itself.

I got the idea; thanks @Simon Pearce and @Niklas Schandry for the help. I will try to only catch really non-sense parameters in case they break the entire workflow and possible warning the user.

While I think this is a noble idea, pipeline developers should be able to figure out if they broke a module via

ext.args

rather quickly, and I guess you can also expect people to think a bit when implementing things in their pipelines 🙂 My main concern is that maybe being to restrictive here restricts some valid use-case. If this applies to the

--temp

flag in the module you are developing I of course do not know.

Yes, normally we were doing that; the problem in our case is that part of our workflow is the parallelization of one big tool that everyone knows in the field of proteomics, which, if you run it single node, has a parameter, for example

--parallize

. In our pipeline, we took different parameters from each step and did the parallelization in nextflow. Now users want to know that the parameters from the sequential runs are trying to use

ext.args

into some of our steps that don't make sense; we controlled by params what should be provide but if we don't control (as you rightfully said); things will break. For expert users is fine; but I want to have an extra validation step to control a bit the chaos for non-experts. But I agree with you that we need to do this only in minimun cases.

Do I understand correctly: You took on the effort of making some tool parallel via nextflow, and now users are unwilling to read the documentation and inject random stuff into the workflow via

ext.args

?

When you maintain a tool, that is used by other sometimes is sometimes good to put some warnings to inform how much freedom they have, to avoid the possibility that things do not become an issue; better to capture it. But agree with you in principle; ext arguments are and should be for freedom.

Well, as I said I think it is very nice of you to try to provide those guardrails.