If I’m using a secured Kafka using SASL_SSL. Is th...
# troubleshootingc
Carlos Domínguez
07/08/2021, 9:41 PMIf I’m using a secured Kafka using SASL_SSL. Is there any way of configuring that and use those credentials? Or there is another way of setting security from Pinot to Kafka for data ingestion?
j
Jackie
07/08/2021, 10:00 PM@Alexander Pucher Can you please help answering this?
Jackie
07/08/2021, 10:03 PMnvm.. Xiang already replied:
I think we have an issue open for SASL_SSL support: https://github.com/apache/incubator-pinot/issues/7134
r
RK
07/09/2021, 10:09 AM@Carlos Domínguez @Jackie JAVA_OPTS="-Xms1G -Xmx4G -Dplugins.dir=plugins -Djava.security.auth.login.config=/home/dev/client_jaas.conf" bin/pinot-admin.sh StartController -configFileName ~/temp/pinot/pinot-test/controller.conf
Use this java option and give ur Jaas file location if u are starting ur component separately if you are using quickstart-batch.sh file then just give Jaas file location as mentioned in this PR.
I created this PR it's working fine now ( answer provided by @Xiang Fu in this PR comment section). Please try that.
c
Carlos Domínguez
07/09/2021, 3:25 PMRegarding security is there a way of specifying credentials from
streamConfigCarlos Domínguez
07/09/2021, 3:25 PMThanks in advance
Carlos Domínguez
07/09/2021, 3:26 PM@Alexander Pucher @Jackie @RK
r
RK
07/09/2021, 3:27 PMsecurity.protocol=SASL_PLAINTEXT (or SASL_SSL or PLAINTEXT)
sasl.kerberos.service.name="kafka"
into your stream configs in your table conf
👍 2
j
Jackie
07/09/2021, 5:54 PMThanks @RK for answering the question. @Alexander Pucher @Xiang Fu Let's add these steps into the pinot doc. Seems a common issue when people trying to enable authorization
x
Xiang Fu
07/09/2021, 6:33 PM
Yes, we can add a section here for how to connect to secure Kafka based on different protocols https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion/import-from-apache-kafka#add-sample-data-to-the-kafka-topic
➕ 3
r
RK
07/10/2021, 2:27 AM@Xiang Fu @Jackie @Carlos Domínguez Updated steps to enable kerberous authentication in this PR.
Thanks.
RK
RK
07/13/2021, 5:30 AM@Carlos Domínguez is it worked for you. (Secured Kafka integration)?
c
Carlos Domínguez
07/13/2021, 5:45 AMNot yet
Carlos Domínguez
07/13/2021, 5:45 AMThe thing is, I’m getting my credentials from Confluent Cloud
Carlos Domínguez
07/13/2021, 5:45 AMAPI KEY and API SECRET
Carlos Domínguez
07/13/2021, 5:46 AMThen I have
Carlos Domínguez
07/13/2021, 5:46 AM Copy code
bootstrap.servers=XXXX
security.protocol=SASL_SSL
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='{{ CLUSTER_API_KEY }}' password='{{ CLUSTER_API_SECRET }}';
sasl.mechanism=PLAINCarlos Domínguez
07/13/2021, 5:47 AMIs there any way of using all those properties in table definition directly?
Carlos Domínguez
07/13/2021, 5:47 AM@RK
r
RK
07/13/2021, 5:51 AMYes we can set bootstrap.servers , security.protocol and sasl.mechanism in table-config file.not sure about Jaas.config , username, password.
@Xiang Fu can suggest.
x
Xiang Fu
07/13/2021, 6:21 AM
Can you try to put them into stream configs section?
c
Carlos Domínguez
07/13/2021, 7:14 AMIt seems its working
👍 1