I'm seeing odd behaviour when running fetchngs in combination with my institutional config file. Somehow, some of the config params are being recognised as pipeline params, and this causes fetchngs to stop:

WARN: The following invalid input values have been detected:

* --vsc_calcua_scratchdir: /scratch/antwerpen/203/vsc20380
* --vsc_calcua_apptainer_tmpdir: /scratch/antwerpen/203/vsc20380/apptainer/tmp
* --vsc_calcua_apptainer_cachedir: /scratch/antwerpen/203/vsc20380/apptainer/cache
* --vsc_calcua_nxf_apptainer_cachedir: /scratch/antwerpen/203/vsc20380/apptainer/nextflow_cache
* --vsc_calcua_partition: zen2
* --vsc_calcua_slurm_scheduling: true

This is the config in question: https://nf-co.re/configs/vsc_calcua/ Oddly enough, I don't see this problem in my own pipeline nor for nf-core/rnaseq. I'm using

NXF_SYNTAX_PARSER=v1 NXF_VER=26.04.4

. Any ideas?

This is due to work to make the config portable with the v2 parser and what I guess is fetchngs not being updated in a long while (and having an older version of what is now nf-schema that looks at params to ignore differently. I can't check the exact name now but some config should have the old name still, and the solution is to add the ignore-params to that.

Oh! Could this be it?

UTILS_NFVALIDATION_PLUGIN

https://github.com/nf-core/fetchngs/blob/master/subworkflows/nf-core/utils_nfvalidation_plugin/main.nf Based on this old blog post (https://seqera.io/blog/nf-schema/), and this API doc (https://nf-co.re/docs/nf-core-tools/api_reference/2.13/pipeline_lint_tests/nextflow_config), that'd mean I have to use an actually parameter called

validationSchemaIgnoreParams

. I'll give it a try.

That seems about right, yes.

Cheers. Will report back later. And thanks, I'd never have found this on my own. Would it be worth documenting this somewhere as a warning? O

"Would it be worth documenting" is extremely seldom something that should get a "no" answer. The problem is rather where that should be 🙂 Except for bandwidth and energy, it would be nice for lint for this. For now, maybe make sure that nf-schema highlights its heritage and old configuration possibly needed as well? (I don't recall how much will be shown in the full message and how far a google search or AI response would get you.)

Seems like a pragmatic solution for now! Who should we ping so that this gets put on a todo or wish list somewhere? I guess @Nicolas Vannieuwkerke since IIRC he did a crazy amount of work on both nf-schema and migrating these institutional configs.

Actually, now when I had a few seconds to look at it, I don't see the things I'd expect in fetchngs for this to blow up (like https://github.com/nextflow-io/nf-validation/blob/master/docs/parameters/validation.md#failing-for-unrecognized-parameters), so coming back to this, are you sure that's what's causing the actual failure?

I'll try to run some more tests to make sure I'm not missing anything.

Yeah unrecognized parameters shouldn't fail the pipeline (unless specifically configured to do so)

Thanks for the confirmation! I didn't get a chance yet to retry unfortunately...

Okay keep me updated 😛