Hi everyone blob wave I created a new PR that adds support for ingesting BigQueryAuditMetadata in the

bigquery

ingestion module (https://github.com/linkedin/datahub/pull/3680). When trying to use the

bigquery

ingestion module, I’ve noticed that computing lineage almost always fails for big projects due to timeouts in the Cloud Logging API. As a workaround, in this PR, I updated the module so that query job completions logs can be extracted from exported BigQuery audit logs (https://cloud.google.com/bigquery/docs/reference/auditlogs#defining_a_bigquery_log_sink_using_gcloud) which should be more stable than accessing the Cloud Logging API for many cases. Let me know what you think. Thanks!

Thanks for this contribution @wooden-gpu-7761!

Woohoo, thanks so much for contributing back to the community @wooden-gpu-7761!!