hi guys
i have multiple kafka sasl separate Kerb...
# troubleshootingb
Bordin Suwannatri
03/28/2022, 3:56 PMhi guys
i have multiple kafka sasl separate Kerberos. i don't know what parameter on real time table use for point to krb5.conf or content inside krb5.conf.
i need to config realtime tables and multiple kdc. Please recommend which parameter or some example use for that.
m
Mayank
03/28/2022, 4:52 PM
@User any suggestions ^^
a
Alexander Pucher
03/28/2022, 5:03 PMthis one is for @User
c
Chinmay Soman
03/28/2022, 5:12 PMI haven’t played around with Kerberos config in Pinot myself . Can’t give any quick answer I’m afraid but can you share the error you’re seeing ? Also I’m assuming you’ve seen this for Kafka client config : https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_gssapi.html#clients
Chinmay Soman
03/28/2022, 5:12 PMIf you have a working Kafka client the same config should work with Pinot
Chinmay Soman
03/28/2022, 5:13 PMMy suggestion is to get the client working first
b
Bordin Suwannatri
03/29/2022, 8:01 AMthis is my config that working with one kdc (/etc/krb5.conf) but i need to config new table consume to another kerberlos to. i don't know parameter how to point to krb5.conf on table config @User.
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.topic.name": "PROD-json-MB-Postpaid-Sales-Online",
"stream.kafka.broker.list": "tykbpr019092,tykbpr029092,tykbpr01:9092",
"stream.kafka.consumer.type": "lowlevel",
"stream.kafka.consumer.prop.auto.offset.reset": "smallest",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"realtime.segment.flush.threshold.rows": "0",
"realtime.segment.flush.threshold.time": "1h",
"realtime.segment.flush.segment.size": "100m",
"stream.kafka.consumer.group.id": "rdp_lookup",
"security.protocol": "SASL_SSL",
"sasl.mechanism": "GSSAPI",
"sasl.kerberos.service.name": "kafka",
"ssl.truststore.location": "/data/apache-pinot/keytab/tykbpr.client.truststore.jks",
"ssl.truststore.password": "xxxx,
"sasl.jaas.config": "com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true useKeyTab=true storeKey=true keyTab=\"/data/apache-pinot/keytab/U-SVC-RDP.keytab\" principal=\"U-SVC-RDP@TRUE.TH\" doNotPrompt=false;"
}
Bordin Suwannatri
03/29/2022, 8:01 AMi'll check your link that you recommend.
Bordin Suwannatri
03/29/2022, 8:03 AMcan i put everyhing to this parameter ? sasl.jaas.config=
Bordin Suwannatri
03/29/2022, 8:05 AMcat /etc/krb5.conf
includedir /etc/krb5.conf.d/
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = TRUE.TH
kdc_timesync = 1
ticket_lifetime = 7d
renew_lifetime = 15d
[realms]
TRUE.TH = {
admin_server = true.th
kdc = true.th
}
TRUE.CARE = {
kdc = smbadpr1b
admin_server = true.care
}
KAFKA.SECURE = {
admin_server = napat-test
kdc = napat-test
}
Bordin Suwannatri
03/29/2022, 9:26 AMi try to use this for point krb5.conf "security.krb5.config": "/etc/krb5truecare.conf"
it's not working.
c
Chinmay Soman
03/29/2022, 5:37 PM@User I really think this is a Kafka question.
Chinmay Soman
03/29/2022, 5:38 PMwe need to identify the right JAAS config that can accept a custom krb5.conf path
Chinmay Soman
03/29/2022, 5:38 PMinstead of default in "/etc/"
Chinmay Soman
03/29/2022, 5:45 PMI actually googled aroudn but couldn't find it - the only mention is the java argument `
Copy code
-Djava.security.krb5.confb
Bordin Suwannatri
03/30/2022, 3:44 AMyes if i set it on java it only 1 krb5.conf
Bordin Suwannatri
03/30/2022, 3:44 AMi have multiple kdc
Bordin Suwannatri
03/30/2022, 6:57 AMPinot support multiple Kerberos for one pinot cluster or not ? @User
Bordin Suwannatri
03/30/2022, 7:01 AM Copy code
{
"_code": 500,
"_error": "org.apache.kafka.common.errors.SaslAuthenticationException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state."
}