Hi Team, lineage got wiped out after upgrading datahub to v0.10.0.

How's your datahub deployed ? If you're deploying the Docker containers yourself (without Helm or Docker-Compose Quickstart), you may want to verify you've run system upgrade as mentioned here https://github.com/datahub-project/datahub/releases/tag/v0.10.0

Datahub is running in kubernetes

How about dataset, is wiped out as well?

NO, only the lineage and glossary terms

That's strange. Are you able to identify the missing lineage - probably check whether upstreamLineage aspect is present for that entity using Datahub CLI ? Is missing lineage particular to an edge/platform or all the lineage edges went missing ?

Hi, the lineage got wiped out for snowflake only. We are not able to get the lineage after re-running or refreshing (ingesting metadata) those wiped-out lineage tables also we tried to ingest new tables and still we cannot figure out why we can't see the lineage to these new tables even. Someone help me out on this, Thanks in advance

what was your older version you upgraded from ? Here are some things we can do to find out the root cause - 1. Confirm lineage extraction is enabled in the recipe and the necessary permissions are granted on snowflake. 2. Confirm that the snowflake ingestion succeeds without any warnings/errors. 3. Run snowflake recipe with file sink and confirm if upstreamLineage aspect is present in the output file

Older Version: 0.9.5 1. We are able to see the lineage in databricks but not in snowflake. 2. I am using sysadmin role while ingesting metadata and it has all the permissions. My question is how can I see if my snowflake is able to allow lineage in datahub or not. 3. grant operate, usage on warehouse POC_DATAHUB_GX_WH to role sysadmin; 4. 5. grant usage on DATABASE POC_DATAHUB_GX_DB to role sysadmin; 6. 7. grant usage on DATABASE ADQT_METADATA to role sysadmin; 8. 9. grant usage on all schemas in database POC_DATAHUB_GX_DB to role sysadmin; 10. 11. grant usage on future schemas in database POC_DATAHUB_GX_DB to role sysadmin; 12. 13. grant references on all tables in database POC_DATAHUB_GX_DB to role sysadmin; 14. 15. grant references on future tables in database POC_DATAHUB_GX_DB to role sysadmin; 16. 17. grant references on all external tables in database POC_DATAHUB_GX_DB to role sysadmin; 18. 19. grant references on future external tables in database POC_DATAHUB_GX_DB to role sysadmin; 20. 21. grant references on all views in database POC_DATAHUB_GX_DB to role sysadmin; 22. 23. grant references on future views in database POC_DATAHUB_GX_DB to role sysadmin; 24. 25. // If you ARE using Snowflake Profiling or Classification feature: Grant select privileges to your tables 26. grant select on all tables in database POC_DATAHUB_GX_DB to role sysadmin; 27. 28. grant select on future tables in database POC_DATAHUB_GX_DB to role sysadmin; 29. 30. grant select on all external tables in database POC_DATAHUB_GX_DB to role sysadmin; 31. 32. grant select on future external tables in database POC_DATAHUB_GX_DB to role sysadmin; 33. 34. // Create a new DataHub user and assign the DataHub role to it 35. create or replace user datahub_user display_name = 'DataHub' password='' default_role = sysadmin default_warehouse = 'POC_DATAHUB_GX_WH'; 36. 37. // Grant the datahub_role to the new DataHub user. 38. grant role sysadmin to user datahub_user; 39. 40. grant usage on schema POC_DATAHUB_GX_DB.RAW to role sysadmin; 41. Let me know if I have to give any more permissions 42. I am not able to see the upstreamlineage in sink file.

For lineage - this is required :

_grant_ imported _privileges_

on

_database_ snowflake _to_ role <rolename>;

Thanks a lot, I am able to get the lineage

I don't know about databricks viz.