Hi! How to remove wrongly ingested lineage? I wrot...
# ingestion
m
Hi! How to remove wrongly ingested lineage? I wrote the lineage file with .yml extension but I added some wrong upstreams by mistake and now I can't undo or delete.
g
You can use
datahub ingest rollback
. The
datahub ingest list-runs
will give you the run IDs that you need
m
Thanks for the feedback Harshal Sheth. But when ingestion is done by python script via cli? Ingestion is not recorded. I have two machines running Datahub in one I ingested it via .yml file and the other by python script via cli.
g
If you ran
datahub ingest -c <recipe.yml>
, then it should show up in the
list-runs
- if not, then we have a bug
m
Sorry for my mess. I made the ingestion using python script as it is in the link: https://github.com/datahub-project/datahub/blob/1325b8ad71ea687b6b89adda1446a527581b4f42/metadata-ingestion/examples/library/lineage_emitter_rest.py I put make_dataset_urn information as pertinent to my case and run the script by command line $python3 lineage_emitter_rest.py
g
In that case, you can use
datahub delete --urn <urn> --hard --aspect upstreamLineage
ah my bad - it should be
--aspect_name
m
I'm sorry. As I said, I'm working with Datahub on two machines (one for testing and one for the instance). On my local machine the command didn't work due to version incompatibility, but on the instance it worked and solved my problem. Thank you very much.
g
Amazing. Let me know if you run into any other issues