Hi team, are there any plans to support lineage at the column/field level?

I believe this is already possible in the backend, but is not exposed on the UI

ah ok, I was looking at the fine grained lineage examples in https://datahubproject.io/docs/lineage/sample_code to see what it does and how it is represented in the UI. Column to column lineage can get really messy in a diagram. I am interested in allowing a user see what transformations are performed when creating a new column (like in a view). Maybe a better approach would be to show the transformation under 'Schema':

Yeah certainly, we have produced some pretty gnarly visualizations, especially when a table is grouped by multiple columns 😄 Maybe someone from the acryl team can comment on the future of column lineage in the UI as I'm not sure what their plans are

It seems that the fine-grained lineage at field level is used to document lineage, but that level of detail is not exposed in the UI.

cc: @little-megabyte-1074

Hi @ripe-apple-36185! Our near-term goal is to surface column-level lineage in the lineage visualization view; the backend doesn’t support capturing the explicit logic used within the column-to-column relationships, but I really like your suggestion -- If you wouldn’t mind, we’d love to have you open a feature request so we can gauge Community interest!