Hi ALL We just evaluate Pinot, i have few questions on it. 1- Is there any readymade plugin or connector available to ingest data from Kafka to Pinot. 2- As zookeeper is removing from Kafka, is there any plan for remove dependency from Pinot ? 3- Is there any solution for cross data centre replication for DR.

1. Pinot can natively ingest data from Kafka. You just specify the topic coordinates in the table config. 2. The reasons that Kafka is removing ZK do not apply on Pinot. Pinot is very stringent when it comes to reading/writing to ZK that allows us to operate Pinot at huge scale without even stressing ZK. 3. One design pattern is to have independent Pinot clusters and replicate the data (via mirror-maker for example).

Thanks for quick response. @Mayank

Hi, has a new approach emerged in these two years? @Mayank

New approach to what @Benito?

Sorry 🙂 I meant a new approch to replicate Pinot between two data centers. Above, the recommendation is to use mirror-maker. I was thinking if it would be possible to link both Pinot databases to the same Kafka topic in the master node. In case of an outage (master goes down), change that link to Kafka instance running in the secondary node.

If you have access to the same kafka topic, then you can simply point the two pinot tables (in two clusters) to the same topic and ingest from there