A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

I am looking to understand more how Data Hub is built under the hood. And if it can completely replace other metastores such as Hive. Any potential helpful docs on this? Successful migration stories from other data catalogs such as Glue or Hive would be really appreciated :pray:

So far, what I have understood is that it cannot be used as a replacement, and only an enhancement.

For example, if I need to interact with Delta tables through sparkSQL, a data hub catalog will probably not work and I would need a Glue or Hive.

Does that sounds about right? :thinking:

Yes that's broadly right. DataHub is a bit higher-level than glue or hive - we can pull metadata from those systems, so in a sense it's a super-catalog


However, it doesn't imitate the APIs that glue/hive provide, so it can't be used as a drop-in replacement for a metastore