May I ask the advised way to upgrade DataHub properly? For the context, we’ve managed to rebase the project from v0.8.35 to v0.10.1 and are thinking about the rollout plan with migration on both MySQL and ES sides, which is expected to take ~18h. (due to full table scan in MySQL to move policies into ES). And it will be great to get some insights from you. • We are super happy to see the migration job datahub-upgrade is decoupled from GMS Spring bootstrap step, but are wondering what’s the best way to deploy it. For example, say our services are running in k8s without Helm, should we deploy datahub-upgrade as a separate k8s workload resource, or within GMS as a sidecar/init container, or there are other recommended ways? • According to our understanding of how ES re-index works, we expect there will be data loss if we don’t turn off mae consumer. Is that a correct assessment? And if that’s the case and the migration is going to take long, is there a way to reduce the system downtime during the upgrade? • Any other tips?

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!

1. It should be a separate workload, we run it as a single execution job with a higher helm priority. It definitely shouldn't be a sidecar to GMS as it's a run once configuration. Since other services also depend on it executing (if you're running consumers in standalone mode), init container for GMS would not be ideal. 2. The job itself sets write mode to false during the reindex execution, but yes to be safe it is better to turn off the consumers and not run ingestions during the upgrade. There shouldn't be data loss as we have logic built into the upgrade to not move forward if the document counts do not line up, but the upgrade would fail until it's able to successfully execute without documents going in. Time taken for the reindex is highly dependent on size of data, but if you have millions of documents it could potentially take hours. Rough estimates should be ~20min per million documents.

@orange-night-91387 hi,thank you ,We are currently facing upgrading from 0.8.44 to 0.10.0. I have found that the latest version of GMS no longer checks for missing indexes and creates them. Is datahub upgrade currently the only entry point for index creation.1.do I have a way to directly create the latest indexes.? and then re-ingest data.