Hi Everyone, Is there any way to Ingest Data from Kerberous Kafka in Pinot?.I have started Pinot with quick-start-batch.sh file and added jaas file location in quick-start-batch.sh. -Djava.security.auth.login.config="/home/hdfs/client_jaas.conf" \ -Dapp.name="quick-start-batch" \ and it worked as expected.Now I am Starting all Pinot Components seprately with the help of config file and added -Djava.security.auth.login.config="/home/hdfs/client_jaas.conf" in all existing Pinot component files (start-broker.sh,start-controller.sh,start-server.sh) and trying to create table but getting below error message. 2021/06/29 222903.708 INFO [AddTableCommand] [main] {"code":500,"error":"org.apache.kafka.common.KafkaException: Failed to construct kafka consumer"}. Someone Kindly suggest how should i use client_jaas.conf to start pinot in clusterd mode so it can read data from kerberous kafka also. @Xiang Fu @Jackie @Jonathan Meyer

I haven’t tried this before. I thought you can set extra kafka configs in your table config

Okay @Xiang Fu security.protocol is already added in config file.let me add Kerberos service name also

also add those configs into your table config

Looks Like Its Not able to read "`java.security.auth.login.config`" Property from

start-controller.sh

file When I am using same Property in

quick-start-batch.sh

then its working fine, but when I am starting controller independently (without using quick-start-batch.sh file) It's showing below error message in controller log file. "`Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set"` In quick-start-batch.sh

exec "$JAVACMD" $ALL_JAVA_OPTS \
  -classpath "$CLASSPATH" \
  -Djava.security.auth.login.config="/home/dev/client_jaas.conf" \
  -Dapp.name="quick-start-batch" \

command to run: bin/quick-start-batch.sh

`"WORKED AS EXPECTED`" In start-controller.sh

exec "$JAVACMD" $ALL_JAVA_OPTS \
-classpath "$CLASSPATH" \
-Djava.security.auth.login.config="/home/dev/client_jaas.conf" \
-Dapp.name="start-controller" \

command to start:bin/pinot-admin.sh StartController -configFileName /home/dev/Pinot/apache-pinot-incubating-0.7.1-bin/bin/controller.conf
--->Table Add command:
bin/pinot-admin.sh AddTable \
-schemaFile /home/dev/Pinot/transaction_schema.json \
-tableConfigFile /home/dev/Pinot/transaction_realtime_config.json\
-exec

error Message:2021/07/06 211601.653 INFO [AddTableCommand] [main] {"code":500,"error":"org.apache.kafka.common.KafkaException:

Failed to construct kafka consumer"}

In Controller Log File: "`Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set"` Kindly Suggest. @Jackie @Xiang Fu @Ken Krugler @Kishore G @Neha Pawar @Mayank @Uday Vallamsetty @Yupeng Fu

@Xiang Fu Is this related to your pinot-admin fix?

this is 0.7.1, so might be another issue

@Jackie @Xiang Fu but this auth is supporting Nd able to read JAAS file when I am using quick-start-batch.sh file in 0.7.1 ?

@Xiang Fu this is JAAS auth. Doesn’t have anything to do with the REST API auth we added recently

Hi Everyone, As suggested by @Kishore G I have raised a PR for the same. https://github.com/apache/incubator-pinot/issues/7134#issue-939150344 Cc: @Xiang Fu @Jackie @Mayank Thanks.