Hi guys. I'm working with your Java lib datahub-client:0.8.34 Now I want to create and push MCP event with dataset schema, and this schema is stored in Avro .avsc files. I can read this files as string, or convert them to org.apache.avro.Schema object, but don't know how to convert it to Datahub's schema. I found some of your avro-to-pegasus translator https://linkedin.github.io/rest.li/avro_translation tried to use it and seems it's working, but I got conflicts with datahub-client library. io.acryl:datahub-client and com.linkedin.pegasus:data-avro - both have com.linkedin.data module with DataShema class. They look similar, but a little bit different. How I can convert Avro schema to the correct Datahub's schema and push it using your Java lib?

Hi! You shouldn't need to do any conversion from Avro to PDL or anything like that. What you'll want to do is essentially create a custom source that takes in an Avro Schema and converts it to a Dataset in a similar way that our current ingestion sources do. Take a look at how the SchemaMetadata object is constructed in: https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/sql/sql_common.py#L869 See below for details on using a custom source: https://datahubproject.io/docs/how/add-custom-ingestion-source/

Yeah, thanks a lot. I know how it was implemented in Python lib, but want to implement the same using Java lib. There I found the same classes for different aspects, one of them is metadata. And yes I can create metadata with schema manually, adding columns one by one, but I have Avro schema, which can be string or apache Avro Schema object, and for me interesting - is it possible to convert somehow this schema to Datahub's dataset schema

You should be able to loop through the fields on the Schema object and convert them to SchemaFields to be put into SchemaMetadata. I don't think you will need any Rest.li libraries to do this, just Avro.