A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hey guys, love the new version.

I like the fact the platforms also show up on the main screen, but it doesn't seem to catch my custom ones.

E.g. Along Tableau Charts, I also have some Zeppelin based-charts. And would like to see Zeppelin popping-up as a platform, too.

The same under datasets. Custom platforms don't show up.

We filter out platforms without logos. Once you ingest a data platform entity for your custom platforms, it should start showing in the module!

Take a look at the data platforms we ingest for you during start up. <https://github.com/linkedin/datahub/blob/master/metadata-service/war/src/main/resources/boot/data_platforms.json> You can mimic one of these for your custom platform like Zeppelin (logo is required) and ingest it!

image.png

Was just about to reply, it was this :slightly_smiling_face:

Indeed after adding my data platforms, they show up.

<@U01T9TUS3NH> would love to get the Tableau connector contribution from you!

<@U01C3DMG2GL>, I am not doing anything special. I have tried lots of things to get the metadata from Tableau, eventually ended up writing my own parser for the cases that we mostly use on Tableau.

Their SDK is not exposing the underlying entities' metadata, but could possibly be used to retrieve the workbooks.

The way I did it, I am running a parser on a bunch of Tableau workbooks and extract the sources from the XMLs (as workbooks are XMLs). It's kind of hardcoding for each use case that interest us and not really super accurate, but good enough for us.

I am then building a CSV file which is then converted in a JSON file eventually loaded via datahub ingest.

I don't have issues sharing my code, either.

*prs_extract_tableau_dashboards_metadata* - parser for the workbooks

prs_extract_tableau_dashboards_metadata.py

Our main source is Oracle, so for other sources it might be different. Looking forward to see if you guys can build something generic enough :slightly_smiling_face: Will gladly replace my scripts.

Hey <@U01NN9CTQ01>, just found this thread and was wondering if there is a way to automate adding custom platforms on startup? Maybe copy’ing over some files (`data_platforms.json`) in to gms/frontend docker images would work?

Ah so right now, yes this requires a fork. You can change this file <https://github.com/linkedin/datahub/blob/master/metadata-service/war/src/main/resources/boot/data_platforms.json> and build the docker image. We are currently working on making this easier by supporting plugins. In that framework, we will have the default platforms defined above but the ability to provide a separate overlay yaml file that adds new platforms or updates the default ones.