red-zebra-92204
03/03/2022, 2:47 AMDataHubGraph.get_aspect(aspect="dataJobInfo")
, i'm getting this error: avro.schema.AvroException: ('Datum union type not in schema: %s', None)
The problem lies on the type
field, which should be either AzkabanJobTypeClass
or string
, but it has the value of {"string": "SPARK"}
.
I don't know how to resolve this? Is this related to the deprecation of class AzkabanJobType
?
Code to produce this error:
graph = DataHubGraph(config=DatahubClientConfig(
server='<https://demo.datahubproject.io/api/gms>',
extra_headers={'cookie': f'{cookie}'}
))
job_info = graph.get_aspect(
entity_urn='urn:li:dataJob:(urn:li:dataFlow:(spark,orders_cleanup_flow,PROD),orders_dedupe_job)',
aspect='dataJobInfo',
aspect_type=DataJobInfoClass,
)
Github issue: https://github.com/linkedin/datahub/issues/4289big-carpet-38439
03/03/2022, 5:02 PM