Hi! I've executed the following superset recipe: ...
# ingestion
b
Hi! I've executed the following superset recipe:
Copy code
source:
  type: "superset"
  config:
    username: xxxx
    password: xxxx
    provider: db
    connect_uri: <http://supersetxxxxxx.com>
    env: "DEV"
sink:
  type: "datahub-rest"
  config:
    server: "<http://datahub-datahub-gms.datahub.svc.cluster.local:8080>"
The ingestion was successful. I can see in Datahub the charts, dashboard and lineage. But in the category of "Datasets" datasets do not appear, although I can see them in the lineage (attached image). How can I see these datasets? Do I need any configuration in the recipe? Thanks for any help.
s
This does not add datasets outside lineage. They will need to be invested separately
plus1 1
m
Hi @adventurous-magician-13964, as Aseem said: Superset ingestion will ingest charts, dashboards and “lineage” with datasets (think of these as “shell entities” with just ids that we ingest). However to be able to search for these datasets and see other important metadata about it (like a fully-fleshed out entity), you will need to ingest metadata from the platform that the datasets are housed in (e.g. postgres, redshift, bigquery, hive etc).
plus1 1
b
Thanks @square-activity-64562 and @mammoth-bear-12532 for the help.