We are deploying pinot on k8s cluster using the community provided helm charts. We are using spot instances for the k8s cluster. Do we have to rebalance the cluster to allow re-assignment whenever a pod gets replaced as the spot instances goes away?

Hmm, not sure if using spot instances for pinot components (other than minions) is a good idea.

@abhinav wagle ☝️

If instance names are preserved (like pinot-server-0) you don’t need to rebalance. But you may have other issues like availability. Also, not sure if your automation will reattach the pvc from the previous instance, and/or if there’ll be any under the hood data movement/copy

• instance names are preserved • automation is re-attaching the pvc from the previous instance to the new pod that comes up with the same name • Availability will be affected but we plan on using replication for our tables to reduce its impact. If it matters most use cases are internal dashboards. • I don't follow "under the hood data movement / copy"

Ok, if you understand those caveats and those are fine for your SLA, I guess you can go for it. You can ignore data move/copy, given that your automation will reattach pvc.

Thank you Mayank

Just a disclaimer, I haven’t personally run that, so YMMV.

Haha yes

We will let the community know how it goes 🙂 We do have option to switch out of spot just in case.