https://pinot.apache.org/ logo
a

Amit Chopra

01/14/2021, 9:08 PM
Hi, trying to troubleshoot an issue i am facing. I have a K8S cluster setup with 4 server instances. For the server, i changed replicas to 2 and did helm upgrade. Even though the servers in K8S has reduced from 4 to 2, i still see the deleted ones in bad state in pinot UI. Shouldn’t the deleted servers from K8S be deleted from pinot as well? Secondly, the problem i am facing is that 2 of the segments are mapped to the deleted servers. And now it is not allowing me to drop the server instances manually too. And those 2 segments too are in bad state. Ideas?
n

Neha Pawar

01/14/2021, 9:13 PM
you prolly need to untag the old instances, and do a rebalance. that should move all segments to the live ones, and only then would you be able to delete the instances
@Xiang Fu are there any special considerations for K8 setup other than this?
guide for untag and rebalance ^^
a

Amit Chopra

01/14/2021, 9:24 PM
got it. Thanks
x

Xiang Fu

01/14/2021, 9:33 PM
by removing servers from k8s won't delete the pinot-server automatically
delete pinot servers instances require manual operations
a

Amit Chopra

01/14/2021, 9:33 PM
ok, thanks