Just a question here, how do you guys get lineage with custom sql? I have seen some of the workbook setup in our tableau where analysts had connection to a database and created a custom sql with bunch of joins and CTE statements to back the chart. When i use GraphQL to query this workbook, the

upstreamTables

field is empty, only the

upstreamDatabases

gets populated.

Lineage with custom sql is handled by querying

customSQLTableConnection

https://github.com/datahub-project/datahub/blob/v0.8.31/metadata-ingestion/src/datahub/ingestion/source/tableau.py#L275-L324

Tableau parse only tables from simple queries in custom SQL. You need to write your own SQL parser (for your language) to get all tables used in SQL.

@colossal-easter-99672 thanks for the info. That explains on some complex queries, I don't see upstreamTables returned from GraphQL call.