Hi Team, We are deploying datahub on AWS EKS. We use RDS MySQL and created auth and encryption secrets manually. But whenever we uninstall datahub helm completely and remove pvc. It looses all the metadata. All the existing users are able to login but none of them are shown on UI. PATs are also lost. What is that I am missing here?

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: 1️⃣ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? Yes button 2️⃣ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Yes button

Are you using the prerequisites for ElasticSearch? If so then you are clearing out your indexed data when you delete the deployment. You can restore these indices using this command from the CLI or through the Rest Endpoint it utilizes assuming your MySQL data is still present and you are not somehow blowing that away too with your delete.

Thanks @orange-night-91387 , I am retaining MySQL data. Restore indices definitely helped. I wonder why this is not an implicit process as it require a manual access to system to just redeploy it on k8s.

Typically people either: 1. Don't delete their full deployment and retain the PVCs as the persistent data for ElasticSearch 2. Use a managed ElasticSearch In both of these cases you would not need to do the restore.

Agree! I was hoping that ElasticSearch will be treated as a plane cache and it should be reloaded whenever new deployments happen. I wonder how it will handle upgrades where DB changed. System should handle gracefully situation when elastic search doesn't have any cache.