A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

hey, from what I saw in this lineage video: <https://www.youtube.com/watch?v=rONGpsndzRw> , for Bigquery the lineage (dataset to dataset) should be created automatically. When is this happening? when you ingest data? how is it being refreshed? Thanks

hihi! it should be happening in the "bigquery-usage" connector if im not mistaken, so you'll have to run that recipe!

<@UV14447EU> can keep me honest :slightly_smiling_face:

Actually running the BigQuery recipe will extract lineage as well (`include_table_lineage` is true by default)

<https://datahubproject.io/docs/metadata-ingestion/source_docs/bigquery>

Thanks. include_table_lineage  combined with start date and end date worked!