Hello again! Is there a way to ingest Lineage of a...
# ingestion
h
Hello again! Is there a way to ingest Lineage of a Dashboard via python ingestion framework? DatasetSnapshotClass has got UpstreamLineageClass in its aspects, but DashboardSnapshotClass has no upstreams in aspects.
h
Looks like this might work, but I was looking for a solution using python ingestion framework. Guess I'll have to create jsons via python and then ingest them, that's just not so convenient
s
what are these dashboards in? there is a superset source for ingestion
h
I've got Tableau and I'm ingesting everything manually to provide additional information in description and to create lineages
s
You can use Python ingestion framework. It just won't be using the things given by the recipes. e.g. dataset to dataset lineage has this example https://github.com/linkedin/datahub/blob/master/metadata-ingestion/examples/library/lineage_emitter_rest.py You can run this after installing datahub in your python environment. To create a dashboard class with charts you can take examples from here https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/superset.py#L221 The docs don't have much in terms of using this outside of the recipes but the code is pretty readable and you can take examples from existing code to write your own code, if needed
h
I know how to use Dataset to Dataset lineage. I'm saying that there are not methods for using Dashboards instead of Datasets in Lineages
Currently I'm creating Dashboards as Datasets object to show the lineage, but that is not a good solution. I'm wondering if I missed something in the ingestion framework, but I've read it from cover to cover
s
You are trying to make what kind of lineage? What is the upstream type you wanted to connect? A chart? A dataset?
h
For example, I need a Lineage where Tableau Dashboard is a downstream dependency and several Exasol Datasets are upstream dependencies