A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

I have already checkout the Demo of yours and it's very impressive with DashBoards and Charts. How can i ingest some superset document like this ? That would be greate if  you can give me the an example about ingesting superset

Hey <@U026P1R6BS8>, did you check <https://datahubproject.io/docs/metadata-ingestion/#superset-superset|this part> in the metadata ingestion documentation?

Is there something like dashboard pattern and chart pattern ?

I don’t think so according to source code: <https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/superset.py>

<@U028JK2U492> is right. <@U026P1R6BS8> Would you like to restrict the dashboards ingested? (Since charts are ingested as dependencies of dashboards, you probably don’t want filters on them). 

superset ingestion runs without error  but no dashboard or chart entries pop up in datahub. What am I missing?

it works for me now. I don't remember why though.

<@U026873CA1X> could you share a masked yml file you used for Superset?

<@U021XETPUNT> I am the wrong Felix. Ask <@U022EKKC6Q5>

<@U021XETPUNT>: <@U027B7R23J4> also uses Superset in his company.

<@UV0M2EB8Q> we did not ingest the dashboards and charts in datahub. We have thousands of charts and hundreds of dashboards. We discussed it internally and decided that for now we would prefer to not add them.

But local testing showed dashboards &amp; charts ingested recently as far as I remember. Maybe there was some bug which got fixed in the time since then?

Thanks for the intro <@UV0M2EB8Q>. Hi <@U027B7R23J4> - shall we connect offline? which timezone are you in?

Hi <@U021XETPUNT> I am in IST timezone. What did you want to connect about?

Superset - Datahub integration and in general about some specific challenges…only if you have some time. I am in India too.. <@U027B7R23J4>

image.png

<@U021XETPUNT> I followed quickstart except
```datahub docker ingest-sample-data```
Hosted a local superset at <http://localhost:28080/> and followed superset quickstart to get some data in

Then used this `superset.yaml` file
```source:
  type: superset
  config:
    # Coordinates
    connect_uri: <http://localhost:28080>

    # Credentials
    username: admin
    password: PASSWORD_HERE
    provider: db

sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"```
Ran this command
```datahub ingest -c superset.yaml```
Dashboard and charts are showing up