Hi. I have a problem on Hive where after the tables are moved to a different schema or renamed, I can still see them in Datahub in previous schema /under previous name. I can see both old and new version of the table, the old one does not disappear from the schema after it's been renamed/moved somewhere else. It happens even though the source metadata updates every day. Any ideas what I can look into to fix this?

Hi Nadia, if I am not mistaken that is due to the fact the ingestions are just like a "photo" of your Hive tables and they just sit one on top of each other. If you would want just to have the tables that currently exist in your Hive, I think what you would have to do is to make the ingestion stateful. This could be done by adding the

stateful_ingestion

to your recipe. Here is more info about it.

@microscopic-mechanic-13766 ohh thank you very much! Interesting. I would think that by default data catalog would show the latest version of source metadata. but apparently not. I'll try stateful ingestion