Hi, Team just wanted to ask, are there going to be feature releases soon that will automatically extract and publish all the column level lineage for bigquery tables just by executing a recipe? because currently, I have a java application that parses bigquery audit log and the application parses the queries particularly the create table statements and extracts column level information out of it, we then emit those lineage information to Datahub via the API. Just wanted to know if the application I'm working on is going to be redundant soon? cc : @astonishing-answer-96712 @little-megabyte-1074

@famous-waitress-64616 or @gentle-hamburger-31302 might also be able to speak to this- I’m unsure of the specifics on upcoming updates to BQ

Hi @square-ocean-28447! @gray-shoe-75895 is in the early stages of looking into SQL parsing for BigQuery lineage, but I’m actually not sure that we’ll be able to support it within the open-source version of DataHub (it will likely require offline processing, which we don’t have a clear story for supporting that in OSS). It might be worth the two of you connecting to see if there’s overlap!