A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hi, I’m running DataHub on GKE as a single production instance and I was wondering if we can separate dataset project into something like PRD, DEV on DataHub UI?  so when we test loading metadata into DataHub we can load it to dataset “dev”. Is this possible? if not I’ll deploy another instance as a development one and load metadata to this instance instead or do you have any suggestion for this? Thanks.

there is a ENV variable you can specify in the ingestion recipe to specify which environment the dataset resides in, and subsequently you can filter by environment in search

but there is limited environment choices _PROD, DEV, QA, STAGE_
you can't use your own, it is a known limitation at the moment

you mean ENV var in recipe.yaml if I’m using DataHub CLI right? but currently, I’m loading metadata via OpenAPI Post request, is this possible too?

not familiar with openapi, but i guess its about making sure your URN ends with the correct env , for instance ,PROD and it should work