https://pinot.apache.org/ logo
#general
Title
# general
b

Bharath Sankara

02/02/2021, 6:36 PM
@here - I am evaluating Pinot for my organization.. We will be ingesting Real-time data from Kafka. What schema formats does Pinot support to ingest from Kafka? We have protobuf serialized data, is there a support for that with pinot? Thanks in advance for your help! (I am sure I will have more questions)
w

Will Briggs

02/02/2021, 6:38 PM
I have only used JSON and Avro - it looks like there is an open issue for supporting protobuf: https://github.com/apache/incubator-pinot/issues/5954
However, it does look like there was Protobuf support added in 0.6.0: https://github.com/apache/incubator-pinot/pull/5293
n

Neha Pawar

02/02/2021, 6:39 PM
this protobuf support is only for Offline currently
from kafka we can only do json/avro
b

Bharath Sankara

02/02/2021, 6:40 PM
Thanks @Will Briggs @Neha Pawar
is there a plan to support protos for Pinot realtime ingestion from Kafka @Neha Pawar?
n

Neha Pawar

02/02/2021, 6:41 PM
yes, we plan to
how soon do you need it?
b

Bharath Sankara

02/02/2021, 6:42 PM
any timeline cadence? Just need to factor that into our design... Thanks again
n

Neha Pawar

02/02/2021, 6:42 PM
@Kishore G ^
b

Bharath Sankara

02/02/2021, 6:42 PM
we are actively developing - so sooner I would say..
n

Neha Pawar

02/02/2021, 6:43 PM
also @Kartik Khare ^ who would likely be working on it
b

Bharath Sankara

02/02/2021, 9:02 PM
@Kishore G @Kartik Khare - any timeline for protobuf support for pinot?