Hi folks! I'm having an issue where my Server inst...
# troubleshooting
d
Hi folks! I'm having an issue where my Server instances go "Dead" every now and then, and I'd like to know if there's a way to check this with a single HTTP request to some "health-check endpoint" or similar. More on this thread.
I couldn't find a specific health-check endpoint for the Server instances, but I noticed that the controller seems to mark them as "Dead" if it doesn't find them in the list of LIVEINSTANCES that it gets from ZK; Is it possible, however, to do this assertion with a single HTTP request in such a way that I can use this as part of a liveness probe for Kubernetes?
l
which version of pinot are u using the pinot-server should have a health endpoint and yes you can make it be part of the liveness probe in kubernetes
i think the helm chart has some configuration around it too
d
0.9, I'll check that out then, thanks man!
m
You should set an alert on the
percentOfReplicas
going down below a threshold.
d
Got it, good idea. How can I check that? Also, I noticed that there is a probe already in the Helm chart, but I think we haven't enabled it (
livenessprobeEnabled
or something like that). And it seems like we're running with way too low heap allocation, with Xmx set to 1GB and Xms to 512MB on a server that has 16GB, isn't that too low?