square-activity-64562
08/03/2021, 8:35 AMsquare-activity-64562
08/03/2021, 10:06 AMlineage_mce = MetadataChangeEventClass(
proposedSnapshot=DatasetSnapshotClass(
urn=_get_dataset_urn(downstream_urns),
aspects=[
UpstreamLineageClass(
upstreams=None,
),
],
)
)
emitter = DatahubRestEmitter("<http://datahub-datahub-gms.apps.svc.cluster.local:8080>")
emitter.emit_mce(lineage_mce)
But that did not workbig-carpet-38439
08/04/2021, 3:55 AMsquare-activity-64562
08/04/2021, 5:54 AMMetadataChangeEventClass({'auditHeader': None, 'proposedSnapshot': DataJobSnapshotClass({'urn': 'urn:li:dataJob:(urn:li:dataFlow:(airflow,replace_cta,prod),cta_global_okr_pct_change_id_lineage)', 'aspects': [DataJobInfoClass({'customProperties': {}, 'externalUrl': '<https://DOMAIN_NAME/admin/taskinstance/?flt0_dag_id_equals=replace_cta&flt3_task_id_equals=cta_global_okr_pct_change_id_lineage>', 'name': 'cta_global_okr_pct_change_id_lineage', 'description': None, 'type': 'COMMAND', 'flowUrn': None, 'status': None}), DataJobInputOutputClass({'inputDatasets': ['urn:li:dataset:(urn:li:dataPlatform:athena,analytics.cta_global_okr,PROD)'], 'outputDatasets': ['urn:li:dataset:(urn:li:dataPlatform:athena,analytics.cta_global_okr_mom_pct_change,PROD)'], 'inputDatajobs': []}), StatusClass({'removed': True})]}), 'proposedDelta': None})
square-activity-64562
08/04/2021, 5:58 AMsome-microphone-33485
08/04/2021, 7:03 AMsquare-activity-64562
08/04/2021, 12:57 PMdatahub_datajobindex_v2/_search?pretty=true&size=100
and "removed" : true
is there for the airflow task that I had marked as removed. Searching in datahub_graph_service_v1
elastic search index it seems it was not removed. That should explain why this is still showing up in the graph as well as in the "Task" tab of the Pipeline.big-carpet-38439
08/04/2021, 2:30 PMgreen-football-43791
08/04/2021, 2:53 PMgreen-football-43791
08/04/2021, 2:53 PMdazzling-appointment-34954
01/19/2022, 1:26 PMdazzling-appointment-34954
01/19/2022, 1:35 PMbig-carpet-38439
01/19/2022, 10:52 PMearly-lamp-41924
01/19/2022, 11:23 PM