Hello! Could you tell me please how to debag avro...
# getting-started
c
Hello! Could you tell me please how to debag avro.schema.AvroException: ('Datum union type not in schema: %s', 'com.linkedin.pegasus2avro.dataset.DownstreamLineage')
g
Hey @cuddly-lunch-28022 - this is saying DownstreamLineage is not an aspect type- and it is correct. The lineage aspect type is UpstreamLineage- downstream lineage is inferred from upstream.
c
sorry i don't understand ) here my script { "com.linkedin.pegasus2avro.dataset.UpstreamLineage": { "upstreams": [ { "auditStamp": { "time": 1581407189000, "actor": "urnlicorpuser:jdoe", "impersonator": null }, "dataset": "urnlidataset:(urnlidataPlatform:hdfs,x,PROD)", "type": "TRANSFORMED" } ] } },{ "com.linkedin.pegasus2avro.dataset.DownstreamLineage": { "downstreams": [ { "auditStamp": { "time": 1581407189000, "actor": "urnlicorpuser:jdoe", "impersonator": null }, "dataset": "urnlidataset:(urnlidataPlatform:hdfs,y,PROD)", "type": "TRANSFORMED" } ] } }
g
Got it- @cuddly-lunch-28022 you shouldn't create the DownstreamLineage aspect
you should just emit upstream lineage 🙂
we then infer downstream lineage on the read side
so you only write upstream lineage- then read upstream & downstream