<https://datahubspace.slack.com/archives/CUMUWQU66...
# ingestion
w
https://datahubspace.slack.com/archives/CUMUWQU66/p1637644851019500 Shedding light on this again and would be great if I could receive feedback! 🙏
l
@wooden-gpu-7761 1) Could you please write up a short doc on the setup required for the workaround i.e how to setup exported audit logs. Is this something that is turned on by default or would require the admin to turn on something? 2) This sounds like a good idea.
Would love a contribution from you since a few people in the community have already encountered the API timeout issue you've mentioned
This is on our radar too but we're currently tackling a few other things.
Please let us know if you can get to it in the next week or so. If not, we can address it after next week
w
@loud-island-88694 yeah I’ll write a doc soon! On 1) you’d have to configure a Logs Router and make sure that a projec’ts BigQuery AuditLogs are all exported to a single partitioned table continually. In my case there are two main execution projects and the AuditLogs of both projects are being exported to their respective tables. 2) Thanks!
I can absolutely get to it next week. What do you think about a PR that addresses both 1 and 2?
l
that will be fantastic
w
https://github.com/linkedin/datahub/pull/3680 I’ve added a PR to support • exported bigquery logs to compute lineage information • multiple query projects for the
bigquery
ingestion source Let me know what you think!