A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

I have a reverse question to this one <https://datahubspace.slack.com/archives/CV2KB471C/p1650645167453159> -- is there a way to hide the "Dataset" nodes (or better yet, not generate them at all)?

They duplicate most of the information included in dbt datasets but are missing node type information (in node vizualization) and view definition (in dataset details), so they're just cluttering the lineage and the UI.

I feel like I'm missing something, why are they even generated in the first place?

Hi Michal, BigQuery nodes have information like profiling, usage etc. While dbt nodes have the logic information and often have additional documentation on the schema etc.

the latest quickstart images include a new feature that merges the view for these nodes while keeping them separate in the underlying metadata graph

Screen Shot 2022-06-24 at 10.44.13 AM.png

Screen Shot 2022-06-24 at 10.43.14 AM.png

Screen Shot 2022-06-24 at 10.42.43 AM.png

here are a few screenshots of the new experience

I work with Databricks but this is very cool, will this "node merging" cover all platforms or only BigQuery for now?

image.png

Hi again :slightly_smiling_face: could you just tell me how do I turn this functionality on? Because I still see separate nodes on 0.8.39: