Ken Youens-Clark
03/19/2025, 8:33 PMcpus value to indicate 32 cores for the "HPC" profile, but it didn't seem to work. Then my workmate pointed out it should be process.cpus . I was surprised that Nextflow wouldn't complain about an obviously bad configuration, so I added this.doesnt.exist = true and, yep, Nextflow accepts this just fine. Can anyone explain why?Phil Ewels
03/19/2025, 9:26 PMNXF_EDGE=1 nextflow self-update
export NXF_SYNTAX_PARSER=v2
And run again.. You should now see a warning in the terminal:
WARN: Unrecognized config option 'cpus'Phil Ewels
03/19/2025, 9:27 PMPhil Ewels
03/19/2025, 9:27 PMPhil Ewels
03/19/2025, 9:28 PMPhil Ewels
03/19/2025, 9:29 PMPhil Ewels
03/19/2025, 9:29 PMNXF_SYNTAX_PARSER=v2 thing is above.Phil Ewels
03/19/2025, 9:30 PMKen Youens-Clark
03/19/2025, 10:00 PMThomas Danhorn
03/19/2025, 11:21 PMwithName, and that would likely be ugly and hard to read. And if something is a simple setting, you can just make a parameter for it, and -params-file already supports JSON.Phil Ewels
03/19/2025, 11:27 PMPhil Ewels
03/19/2025, 11:28 PMPhil Ewels
03/19/2025, 11:28 PMPhil Ewels
03/19/2025, 11:29 PMPhil Ewels
03/19/2025, 11:29 PM-params-file for a long time.Nicolas Vannieuwkerke
03/20/2025, 8:27 AMPhil Ewels
03/20/2025, 2:49 PMNicolas Vannieuwkerke
03/20/2025, 2:54 PMPhil Ewels
03/20/2025, 2:56 PMPhil Ewels
03/20/2025, 2:57 PMNicolas Vannieuwkerke
03/20/2025, 2:58 PMNicolas Vannieuwkerke
03/20/2025, 2:58 PMDoubt this will be default until 25.10, would be my guessI hope so 😅
Ken Youens-Clark
03/20/2025, 4:58 PMcontainer directive in my NF script (not the config) after the output declaration and it was therefore seen not as the Docker image to use but some invalid part of my output and so it was silently ignored and my process wasn't running inside my container. Wow, that's bad.Ken Youens-Clark
03/20/2025, 5:00 PMinput: and output: blocks, leading me (a new NF user) to assume there's some sort of white-space parsing of block structure a la Python/Haskell, but that's not the case. Curious, why didn't these use {} blocks like elsewhere? Seems exceedingly dangerous.Ken Youens-Clark
03/20/2025, 5:09 PMKen Youens-Clark
03/20/2025, 5:26 PMpublishDir 'results', mode: 'xyz' as a compilation error but it's still a runtime error with "No enum constant nextflow.processor.PublishDir.Mode.XYZ". Seems like that would be a good thing to parse ahead of time?Ken Youens-Clark
03/20/2025, 5:29 PMpath 'out/components/component-*', emit: components
But putting the "emit" argument in quotes is not:
path 'out/components/component-*', emit: 'components'
But the error message is "ERROR ~ sculu.nf at 10, 9: Unrecognized output components for process `make_components`" without telling me what the problem is.Ken Youens-Clark
03/20/2025, 5:40 PMpath 'foo', emit: true
This is in my "make_components" process, and the error message I get is: "Process 'make_components' has been already used -- If you need to reuse the same component, include it with a different name or include it in a different workflow context". That makes no sense!Ken Youens-Clark
03/20/2025, 5:42 PMKen Youens-Clark
03/20/2025, 5:44 PMpath 'foo', emit: foo, optional: yes gives me the error "ERROR ~ sculu.nf at 24, 42: yes is not defined"Thomas Danhorn
03/20/2025, 5:56 PMyes as true or false, it should work.Ken Youens-Clark
03/20/2025, 6:06 PMpath 'foo', emit: foo, optional: yes, "foo" is treated as a string and "yes" is treated as a variable but there's no discernible difference for the user.Ken Youens-Clark
03/20/2025, 6:07 PMKen Youens-Clark
03/20/2025, 6:15 PMyes = true in my code and then write path 'foo', emit: yes, optional: yes and I guess that means it will be an optional output that can be found in process.out.true. Yikes.Thomas Danhorn
03/20/2025, 6:34 PMtrue), and anything quoted as a string. Since types are optional in Groovy and basically discouraged/forbidden in Nextflow, you can assign a boolean to a string without error, and the language will try to do the most sensible thing at its disposal, e.g. a string with value 'true'. Of course if you use a bareword that turns out to be a variable, and you are unaware of its content, you might be in for a surprise.
I think at the root of the issue is the lack of strong typing (in C assigning a string to number or boolean or vice versa just can't be done, except maybe with very explicit casting), but that is the very thing that makes scripting languages very powerful and convenient. (You probably would not be happy if you had to develop and compile a C program with variable declarations for every 10-line Python you whipped up to do something real quick.) Definitely a double-edged sword.Ken Youens-Clark
03/20/2025, 6:53 PMPhil Ewels
03/20/2025, 7:10 PMPhil Ewels
03/20/2025, 7:11 PMPhil Ewels
03/20/2025, 7:11 PMPhil Ewels
03/20/2025, 7:12 PMPhil Ewels
03/20/2025, 7:13 PMPhil Ewels
03/20/2025, 7:14 PMPhil Ewels
03/20/2025, 7:15 PMPhil Ewels
03/20/2025, 7:15 PMBen Sherman
03/20/2025, 7:32 PMKen Youens-Clark
03/20/2025, 8:11 PMcontainer directive/declaration(?) after the output: and the v1 parser silently ignores it and runs the program in my current environment. I didn't realize the problem because I have the same program installed locally.