Hello, We’re running `0.9.6.1` on EKS with managed...
# troubleshoot
b
Hello, We’re running
0.9.6.1
on EKS with managed storages (RDS, OpenSearch, MSK). After an attempt to upgrade to
0.10.2
, we had to perform a disaster recovery on OpenSearch cluster, as
datahub-upgrade
pod failed along with
datahub-gms
pod and we couldn’t restore the functionality with rollback. All upgrade jobs were set to
true
before applying the Helm chart. Chart itself was updated before an upgrade with
helm repo update
. Questions: • Is there any breaking change we’re not aware of (like too many major versions skip)? We did multiple upgrades in the past and this is the first time we’re blocked. • Shall we perform a manual diff on
values.yaml
to mirror GitHub’s chart to-the-letter?
🔍 1
📖 1
✅ 1
l
Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!
a
Hi, @brainy-tent-14503 might be able to help you get some insight here, This is likely due to the reindex that would have occured during the upgrade to 0.10, which can take a long time depending on the amount of data you have
b
Can you share the logs from the
datahub-upgrade
job pod specifically?
There are a lot of differences between a helm chart for 0.9.6.1 vs 0.10.x for sure. Likely I’d recommend starting with the latest values file and fill in values from your older one.
p
We will post logs on Monday earliest, currently we have around 20k datasets ingested on this env