If a value is a string but can be numeric, e.g.

"1.0"

, what's the best practice?

The best practice is to use YAML or JSON files as they have solid typing built-in. If that's not possible you should allow

"string"

and

"number"

as valid types for the input. Alternatively you can set the validation to lenient mode with

validation.lenientMode = true

in a config but that will make all validation lenient 😛

I think the situation here is someone can have a range of options: • 1.0 • v1.0 • 1.0-beta Sadly, I'd love to enforce strict typing but I don't think that's possible 🙄

Allowing both types and manually converting to a String in the pipeline is probably the easiest option in this case

to be honest, I think that all integers are valid strings

That's what lenient mode does 😉

This isn't about yaml/json btw. The issue exists there too.

nf-schema

uses the

toValue

method of json nodes for converting to the nextflow type object. This will turn anything that looks like an integer into one, regardless of what is was supplied as in the inputs and regardless of what type the schema says it should be. This has been discussed at length multiple times and we've never really settled on a proper fix

Typed params should effectively solve this problem, I'm hoping 👍🏻

Yeah type inferring with complex schemas is just an impossible task... Using JSON/YAML will work since there is no type inferring in nf-schema for those file formats. The plugin can automatically infer the correct type from the file itself

Unless something has changed it still doesn't work I think Nic? Because of the way we cast back to objects for nextflow. I think it works in validation maybe, but then if you assume it's a string in nextflow vs an int you'll get an error

Yes parameters have a completely different set of problems compared to samplesheets. The parameters issues will be solved by the typing, but the samplesheets might need some more work in the future

I'm hoping that samplesheets will benefit from record types, which have type casting

I think I said it on another thread a while back, and I don't want to careen off on a tangent here, but I wonder what the future of validation looks like with record types covering a lot of their scope. We are currently at a halfway house between nf-schema and pydantic it feels like (discriminated unions a big issue for both I think). Not sure if that sort of thing is addressed in the podcast

Yes the future of samplesheets definitely lies withing record types: https://github.com/nextflow-io/nf-schema/issues/196. I'm waiting on an edge release with a stable implementation of the record types to start working on it 😛

100% 👍🏻 It's the natural order of things: key community projects getting absorbed into core Nextflow. Hopefully a win for everyone ❤️

shared a GIF

samplesheetToRecords

sounds good - what I was referring to is more extreme than that even. But I'll leave you in suspense for now so you listen to the next podcast 😆

I'm looking forward to it, the previous one was really interesting. Such a cliffhanger 😉

shared a GIF

I get that 🤣 but such a topic does deserve multiple episodes