https://pinot.apache.org/ logo
#general
Title
# general
a

Avinash Kumar

03/31/2022, 3:51 AM
Hi, I am going to setup apache-pinot for production on AWS EKS and I want to setup multi AZ setup So how would I manage the data because ebs volume support only in single AZ. Can we use efs ? Please suggest Setup should be scaleable and fault tolerance Thanks
m

Mayank

03/31/2022, 5:29 AM
I recommend that a single pinot cluster does not span across AZ, You can replicate the cluster in different AZ by replicating the data pipelines, for DR.
Within AZ, you can still have replication on table for scalability as well as fault tolerance
a

Avinash Kumar

03/31/2022, 7:04 AM
Thanks @User, Is there any document which guide me to setup multi AZ data replication
d

Diogo Baeder

03/31/2022, 12:16 PM
I think he meant replication in a single zone, like on multiple nodes. For multiple zones though you could manually structure your data ingestion to be done at all the desired zones. For example publishing events to multiple Kafka instances.
1
m

Mayank

03/31/2022, 1:43 PM
Yes, that’s what I meant. For streaming, something like mirror maker will replicate Kafka. For offline ingestion, the same pipeline can push data to multiple Pinot clusters in different AZs
cc: @User