Thanks Mars Lan! The town hall meeting sounds good.
If I may ask one question here, it would be about how granular the data lineage is that you track at LinkedIn (or other users). From what I understand, ‘dataset’ is a core entity, and one could use an API call to get ‘upstreams of a dataset’. Do you also try to track field-based lineage? E.g. for one particular column that enters a kafka stream, would you be able to trace down all the tables that later are related to this column? And do you also track transformations to this column while it travels the DAG (e.g. filter for certain values, group by another dimension)?