Hi, I have a problem with duplicates. I don't know...
# troubleshoots
steep-midnight-37232
07/01/2022, 3:25 PMHi, I have a problem with duplicates. I don't know why but I see twice a snowflake database (and its schemas and tables) as you can see in the picture. What should I do? This can lead problems with related lineage. Thanks
s
square-activity-64562
07/01/2022, 4:18 PMCan you copy the URN of both by going to the dataset pages and paste them here? There would be some difference in the URNs
s
steep-midnight-37232
07/01/2022, 4:41 PMok, let me try
steep-midnight-37232
07/01/2022, 4:46 PMSo, the difference in the two URNs is the container
steep-midnight-37232
07/01/2022, 4:47 PMs
square-activity-64562
07/01/2022, 5:05 PMDid you do some kind of migration in your snowflake?
square-activity-64562
07/01/2022, 5:08 PMSuggest that you do a soft delete of everything snowflake as per https://datahubproject.io/docs/how/delete-metadata/#delete-all-datasets-in-the-dev-environment, verify nothing of snowflake is showing up in the UI, and then re-run your snowflake ingestion which should only get you the current ones
square-activity-64562
07/01/2022, 5:09 PMIf you are still getting duplicates then there is something else
square-activity-64562
07/01/2022, 5:09 PMDo you have multiple snowflake accounts?
square-activity-64562
07/01/2022, 5:09 PMwith same database names?
s
steep-midnight-37232
07/01/2022, 5:29 PMI see. the snowflake account is one but i have 2 datahub users with different roles (and grants) in snowflake. The two roles access different databases. No overlap there. So i have 2 recipes for snowflake ingestion, each accessing different databases. Don't know if this could have caused the duplicates. I will remove the metadata as you suggested and let you know if it solves the problem. Thank you very much for your help
3 Views