I put this in office hours, but it probably belongs here: Hello you lovely people, We've been running Datahub for a bit, and I got into an error state today after upgrading Kubernetes (in which it's running). All I did was move the container from an old node version to a new one. It is "up" in running state, but unhealthy. While I could redeploy as we usually do, I'd like to understand how to recover from it in the future instead, to allow for self healing when the health check fails. It starts with: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8080 Caused by: java.net.ConnectException: Connection refused at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:829) 2023-05-19 082228,844 [R2 Nio Event Loop-1-2] WARN c.l.r.t.h.c.c.ChannelPoolLifecycle:139 - Failed to create channel, remote=localhost/127.0.0.1:8080 io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8080 And continues to get connection refused a couple of times. I figured it was a timing problem with the prerequisites, but restarting mysql, kafka and zookeeper did nothing to fix it. I found a thread somewhere on Github to enable Datahub insights, which I have done but with no results. Any ideas what is causing this issue ? Thanks in advance, and have a wonderful weekend if it comes to that. 🙂 Edit: Solved by @damp-insurance-99795,as upgrading to newest (0.24) fixed the issue for me!

I run into a similar issue... The fix for me was to upgrade to the latest helm chart and delete the old

datahub-gms

pod as the

datahub-nocode-migration-job

was running. If you are not using the official helm chart then you should follow this: https://datahubproject.io/docs/advanced/no-code-upgrade/