Hey everyone :wave: We are currently setting up ou...
# generalm
Malte Granderath
04/14/2023, 7:11 AMHey everyone 👋 We are currently setting up our production orchestration for Pinot and we have to decide between focusing on a single cluster vs multi cluster setup. Generally the standard approach at our company for any infrastructure is to go for multi cluster setup because of the increased fault tolerance and less possible cross use-case impact. What are the main advantages of going for a single cluster with multiple tenants approach that are not directly obvious?
k
Kishore G
04/14/2023, 5:56 PM
Efficiency.. operability.. maintenance overhead.. easy to move tables around as the usage grows etc
Kishore G
04/14/2023, 5:57 PM
Just as FYI entire LinkedIn runs on one single 1000+ node cluster across 100+ usecases..
👍 1
Kishore G
04/14/2023, 5:58 PM
Most companies run one cluster per usecase because the underlying system was not built for a multi tenant set up.
m
Mayank
04/15/2023, 4:50 PM
To add, specifically for disaster recovery, you may consider replicating the clusters in different regions.
m
Malte Granderath
04/16/2023, 2:14 PMEfficiencyWhat would be the largest difference between a multi cluster vs single cluster setup for efficiency?
Malte Granderath
04/16/2023, 2:14 PMAs a bit more context we writing a kube operator on top of a bunch of in-house tooling so the maintenance and operability impact between a single or multi cluster setup does not really change a whole lot.
Malte Granderath
04/16/2023, 2:22 PMThe biggest worry for us is that failure of a single ZK cluster can take down essentially all use-cases depending on Pinot
Malte Granderath
04/16/2023, 2:24 PMJust as FYI entire LinkedIn runs on one single 1000+ node clusterWe’d probably be looking at similar numbers (depending on further adoption)
m
Mayank
04/16/2023, 2:27 PM
Yes, in that setup, there is a single cluster per data center. And data pipelines keep the clusters in eventual sync. This not only gives DR, but also avoids any cross region traffic (network latency + cost).