https://pinot.apache.org/ logo
d

Devashish Gupta

02/09/2021, 8:06 AM
Hello Team, I was trying to work with pinot realtime table on Kafka and S3 as deep storage. My pinot-server has this error in the logs : java.lang.ClassNotFoundException: org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory. This setup is done kubernetes via helm. Earlier I was getting a different error around S3PinotFS, to fix that I updated the server config to use pinot-s3 plugin. After that it seems like Kafka plugin is not available to the server. How do I solve this?
x

Xiang Fu

02/09/2021, 8:16 AM
did you set
plugins.dir
and
plugins.include
when you start pinot-server? if so, you need to set
plugins.include
value to
pinot-s3,pinot-kafka-2.0
d

Devashish Gupta

02/09/2021, 9:07 AM
Thanks, it worked. I didnt have to add the kafka plugin before when I was not using s3 as the deepstorage.
x

Xiang Fu

02/09/2021, 6:12 PM
right, by default pinot loads all the plugins if not specified.
also if you are using parquet file, you also need to include
pinot-parquet
d

Devashish Gupta

02/10/2021, 11:00 AM
Thanks, I am using json format for now. Will keep that in mind.