Hello, I’m trying to add/delete lineage with python emitter. Adding and modifying lineage works, but I cannot delete the lineage.

import datahub.emitter.mce_builder as builder
from datahub.emitter.rest_emitter import DatahubRestEmitter

# Construct a lineage object.
lineage_mce = builder.make_lineage_mce(
    [], # Empty upstream dataset to delete the lineage.
    builder.make_dataset_urn("bigquery", "test.TEST_DATASET.dev", "PROD"),
)

# Create an emitter to the GMS REST API.
emitter = DatahubRestEmitter("<http://localhost:8080>")

# Emit metadata!
emitter.emit_mce(lineage_mce)

How can I delete the lineage with python REST Emitter? I’m using the latest(

v0.8.17

) version.

Hey! This should delete lineage. do you see the new version of this aspect in mysql after emitting above? or do you see errors?

@early-lamp-41924 Yep, in mysql the upstream lineage deleted properly, but on UI nothing happens.

@big-carpet-38439 Didn’t we fix this issue before? Setting the array as empty is not removing the edges.

Yes indeed

I’m using Elastic as Graph Store.

Thanks Sunghwan. Let me try to reproduce this on our end and get back

Hi I want the reverse of this, can we not delete the old lineage when we emit a new one?

hey @careful-apple-42677, did you ever find a solution for this? i’d also like to emit new lineage while leaving existing lineage in place

Unfortunately the overwrite is the only option. That being said, we are planning to release some new GraphQL APIs that will make working with lineage much easier. cc @bulky-soccer-26729 !

nice, thanks for the reply @big-carpet-38439! Is there anywhere i can track that?

Hey, is

Empty upstream dataset

(as in the snippet above) still the only option to remove the lineage edge using SDK?