Hi all, is there any way to write code to find column level lineage out of sql queries?
b
big-carpet-38439
11/02/2022, 9:34 PM
There are a variety of SQL parser that you could use. But you'd need to convert into our model as far as I recall. @gray-shoe-75895 Is there a parser we are using internally?
g
gray-shoe-75895
11/02/2022, 9:36 PM
We’re using sqllineage internally, but that only supports table-level lineage. I’m not aware of any open-source sql parsers that support column-level lineage
l
lively-dusk-19162
11/02/2022, 11:55 PM
There is a parser named zetasql, but dont know whether it is a good idea to use that for column level lineage.
b
big-carpet-38439
11/03/2022, 5:00 AM
I see- well if it's helpful we can send some implementation of how we produce the CLL metadata. We can see if perhaps ZetaSQL would be able to help