Hi everyone. We are currently evaluating datahub in our company and I wonder if there is already support or planned support for JSON Schema ingestion? I've seen you support inferring schema information from JSON files, but we already have our schemas defined.
plus1 1
m
mammoth-bear-12532
10/02/2022, 7:12 AM
Are these standalone schemas or attached to a dataset in a datastore (like mongodb)?
r
rich-pager-68736
10/04/2022, 5:50 AM
I'd say they are standalone schemas or at least agnostic to any data store technology. They are, for instance, used for validating messages on SNS/SQS or stored JSON data on S3.
m
mammoth-bear-12532
10/05/2022, 2:50 PM
We definitely want to support standalone JSON schemas (similar to how we have support for protobuf schema repos).
mammoth-bear-12532
10/05/2022, 2:51 PM
Would a solution similar to the protobuf integration we have built work in your case?
r
rich-pager-68736
10/06/2022, 8:00 AM
Yes, I think so. As far as I understood it, you also support ingesting "vanilla" Protobuf schemas, so the datahub meta information is optional? That would work for us in the case of JSON schema files.
Have you already started working on such a feature and if not, what's the anticipated timeline?
m
mammoth-bear-12532
10/07/2022, 4:06 AM
yup that's right, should definitely be done by Q4 this year.