Hi team, I am not able to extract redshift lineage...
# troubleshoot
r
Hi team, I am not able to extract redshift lineage
Copy code
'warnings': {'extract-QUERY_SCAN': ["Error was (psycopg2.errors.FeatureNotSupported) Specified types or functions (one per INFO message) not supported on Redshift tables.\n\n[SQL: \n            select\n                distinct cluster,\n                target_schema,\n                target_table,\n                username,\n                source_schema,\n                source_table\n            from\n                    (\n                select\n                    distinct tbl as target_table_id,\n                    sti.schema as target_schema,\n                    sti.table as target_table,\n                    sti.database as cluster,\n                    query,\n                    starttime\n                from\n                    stl_insert\n                join SVV_TABLE_INFO sti on\n                    sti.table_id = tbl\n                where starttime >= '2023-04-18 00:00:00'\n                and starttime < '2023-04-19 08:00:04'\n                and cluster = 'egateewh'\n                    ) as target_tables\n            join ( (\n                select\n                    pu.usename::varchar(40) as username,\n                    ss.tbl as source_table_id,\n                    sti.schema as source_schema,\n                    sti.table as source_table,\n                    scan_type,\n                    sq.query as query\n                from\n                    (\n                    select\n                        distinct userid,\n                        query,\n                        tbl,\n                        type as scan_type\n                    from\n                        stl_scan\n                ) ss\n                join SVV_TABLE_INFO sti on\n                    sti.table_id = ss.tbl\n                left join pg_user pu on\n                    pu.usesysid = ss.userid\n                left join stl_query sq on\n                    ss.query = sq.query\n                where\n                    pu.usename <> 'rdsdb')\n            ) as source_tables\n                    using (query)\n            where\n                scan_type in (1, 2, 3)\n            order by cluster, target_schema, target_table, starttime asc\n        ]\n(Background on this error at: https://sqlalche.me/e/14/tw8g)"]},
#troubleshoot #ingestion I use
datahub docker quickstart
command to deploy DataHub version v0.10.2, DataHub CLI version: 0.10.2.1, Python version: 3.8.15
✅ 1
🔍 1
📖 1
l
This content can't be displayed.
r
I found a git mr here: https://github.com/datahub-project/datahub/pull/7698, seem like the lastest version v0.10.2 already had figured out extracting redshift lineage issue but not well to my env
b
@rapid-accountant-13127 were you able to resolve?
r
@brainy-balloon-97302 upgrade the lastest datahub cli version then follow this doc https://datahubproject.io/docs/ui-ingestion, change the cli version with the lastest datahub cli version, It works for me.