Hello team, for some tables I am manually using Me...
# ingestion
m
Hello team, for some tables I am manually using MetadataChangeEventClass for ingestion. how do I start debugging MCE format mismatch from AvroTypeException error, I can't seem to find which record has the error. What keyword should I search to find in the error log?
Copy code
avro.io.AvroTypeException: The datum MetadataChangeEventClass({'auditHeader': None, 'proposedSnapshot': ...
Here's a bit context for what I'm doing. I've doing it by trial and error for other aspects, but I'm kind of stuck with this aspect (SchemaMetadataClass)
Copy code
fields = [SchemaFieldClass(...), ...]
schema_metadata = SchemaMetadataClass(fields=fields,...)
mce = MetadataChangeEventClass(aspects=[schema_metadata], ... )
mce = mce.to_obj() 
json.dump(mce, ...)
h
s
The model is explained under Entities tab in docs. e.g. for https://datahubproject.io/docs/generated/metamodel/entities/dataset which might help
m
Thank you everyone, I found out the error is in schemafieldclass. A single field has None type as the description.