Hi there, thanks for creating Hamilton, it's been a blast using it. Before I create a bug report, I have a question about using hamilton pandera dataframe validators with polars. I saw the following linkedin post. It mentions there that using the check_output decorator should is supported for panderas + polars. However I'm getting the following error when running the minimal example found here: https://github.com/jonas-meyer/hamilton_polars_pandera

Actual error: No registered subclass of BaseDefaultValidator is available for arg: schema and type <class 'polars.dataframe.frame.DataFrame'>. This either means (a) this arg-type contribution isn't supported or (b) this has not been added yet (but should be). In the case of (b), we welcome contributions. Get started at <http://github.com/dagworks-inc/hamilton|github.com/dagworks-inc/hamilton>.

I stepped through the following file a bit: https://github.com/DAGWorks-Inc/hamilton/blob/main/hamilton/data_quality/pandera_validators.py#L9 And it seems like the polars plugin isn't part of the supported extensions, I'm assuming this is the issue? Thanks!

Hi Jonas! Happy you're enjoying Hamilton so far! Thanks for bringing this to our attention. I'll take a look right now!

Cheers! Might just be me not understanding the target_ parameter...

You're right, adding

"polars"

to the list of extensions seems to fix the issue. We'll be releasing a patch soon I also noticed that you were using the class-based API with

pandera.polars.DataFrameModel

and

pandera.polars.Field

. It didn't exist when we first built the pandera plugin. Instead you creating

pandera.DataFrameSchema(...)

objects with columns specifications via

pa.Column(...)

It seems to be fully compatible, but let us know if you find some rough edges!

Thanks for the quick reply 🙂 I would be happy to help out adding support for the class based API, is this something that you think is beneficial? Ignore me, misread your message that it wasn't compatible!

yup, it should just work! We're always open to contributions though!