A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

image.png

Hi,  how did we add these hive data source queries here, which I found on demo

refer to line 3125-3140 in <https://github.com/linkedin/datahub/blob/master/metadata-ingestion/examples/mce_files/bootstrap_mce.json>

Exactly - these are hand curated metadata as of today. I’m guessing you’d be interested in a hive-usage connector?

Yes, you're right. I'm really interested in those features. I see this ' datasetUsageStatistics' aspect being used here, but how do we add the data and where do we get the data from   :excited:
Will the data be acquired automatically in the future

:slightly_smiling_face: You'd have to build a hive-usage connector, similar to our <https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/usage/snowflake_usage.py|snowflake-usage> or <https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/usage/bigquery_usage.py|bigquery-usage> connectors to extract the query pattern information out of Hive!