Hello all, I have a question about the ingestion of documentation that we have written in yaml for (bigquery) tables. What would be the best way to enrich the out of the box bigquery ingestion meta data with documentation that we have in version controlled yml files? Example below:

full_name: project-p-p:stats.active_stats
name: active_stats
owners:
  - email: <mailto:abel@vanmoof.com|abel@vanmoof.com>
notes:
description: Collect stats...
usage:
  - department_name:
    example_usage:
      - hello
bigquery_link: <https://bigquery.googleapis.com/bigquery/v2/projects/blabla/datasets/bla/tables/active_stats>
columns:
  - name: frame_number
    description:
    is_primary_key:
    aliases: []
    unit:
    relations: []
  - name: created_at
    description:
    is_primary_key:
    aliases: []
    unit:
    relations: []
  - name: product_id
    description:
    is_primary_key:
    aliases: []
    unit:
    relations: []

@shy-island-99768: this is very interesting. You could use the python sdk to read your yaml file and sling over metadata events to update the documentation of the tables and the fields.

Awesome thanks for the feedback! Already loving datahub 🙂