Hi some of my pinot instances are dead for unknown reasons. Is that any way to recover that? Because queries may fail if it is directed to a dead broker.
I am using the pinot 0.7 docker
m
Mayank
04/20/2021, 5:26 PM
You can restart the nodes to bring them back
However, please also check the logs to see why they went into this state (could be OOM).
n
Neil Teng
04/20/2021, 5:33 PM
docker restart pinot- controller/broker/server does not recover the state.
BTW, where is the log?
I can see in the cluster manager UI that has a error under liveinstance config:
Copy code
{
"code": 404,
"error": "ZKPath /PinotCluster/LIVEINSTANCES/Controller_172.18.0.7_9000 does not exist:"
}
m
Mayank
04/20/2021, 5:44 PM
Logs should be on the instances themselves (whereever the log4j was configured)
The restart should show some info in the logs on why this is happening.
n
Neil Teng
04/20/2021, 6:09 PM
I tailed the log from where it starts to throw exceptions