This message was deleted.
# troubleshootings
Slackbot
01/10/2023, 11:39 AMThis message was deleted.
a
Adam Whitter
01/10/2023, 3:57 PMwhat authentication do you have configured for Kafka? is this kerberos?
What's in your client jaas file? has this worked before and recently stopped or are you setting it up for the first time?
m
MarekM
01/10/2023, 4:51 PMKafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=false
serviceName="kafka"
useKeyTab=true
storeKey=true
keyTab="/etc/security/keytabs/druid.keytab"
principal="druid/xxxxxxxxxxxxxxxxxxx@XXXXX";
};
MarekM
01/10/2023, 4:58 PMIt has a momentary connection to kafka, but there is a problem with the stream
MarekM
01/10/2023, 4:58 PM"status" : "FAILED",
"duration" : 225,
"errorMsg" : "org.apache.kafka.common.KafkaException: Failed to construct kafka consumer\n\tat org.apache.kafka.clie...",
"location" : {
"host" : null,
"port" : -1,
"tlsPort" : -1
MarekM
01/10/2023, 5:04 PMi have many task failed
MarekM
01/10/2023, 5:14 PM"errorMsg": "org.apache.kafka.common.KafkaException: Failed to construct kafka consumer\n\tat org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:830)\n\tat org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:666)\n\tat org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:647)\n\tat org.apache.druid.indexing.kafka.KafkaRecordSupplier.getKafkaConsumer(KafkaRecordSupplier.java:301)\n\tat org.apache.druid.indexing.kafka.KafkaRecordSupplier.<init>(KafkaRecordSupplier.java:71)\n\tat org.apache.druid.indexing.kafka.KafkaIndexTask.newTaskRecordSupplier(KafkaIndexTask.java:102)\n\tat org.apache.druid.indexing.kafka.KafkaIndexTask.newTaskRecordSupplier(KafkaIndexTask.java:37)\n\tat org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:419)\n\tat org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:266)\n\tat org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.runTask(SeekableStreamIndexTask.java:151)\n\tat org.apache.druid.indexing.common.task.AbstractTask.run(AbstractTask.java:169)\n\tat org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:477)\n\tat org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:449)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set\n\tat org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)\n\tat org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)\n\tat org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:82)\n\tat org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:167)\n\tat org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)\n\tat org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)\n\tat org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:738)\n\t... 16 more\n",
"segmentAvailabilityConfirmed": false,
MarekM
01/10/2023, 5:16 PMkafka in version 2.8.2 but client in druid is kafka-clients-3.3.1.jar
a
Adam Whitter
01/10/2023, 7:28 PMIt doesnt look like this is a problem with versions, looks more like a problem with the configuration. The same error you showed originally is in the latest stack: "Could not find a 'KafkaClient' entry in the JAAS configuration" which suggests the jaas file is not being loaded.
Where are you setting java.security.auth.login.config ??
m
MarekM
01/11/2023, 6:57 AMI set the file path in all files: jvm.config
MarekM
01/11/2023, 6:57 AM-Djava.security.auth.login.config=/opt/druid/druid-26.0.0/conf/druid/single-server/micro-quickstart/_common/client_jaas.conf
MarekM
01/11/2023, 7:22 AMThe solution to the problem is to add an entry to the file: runtime.properties in middleManager
MarekM
01/11/2023, 7:22 AMThank you for your help
a
Adam Whitter
01/11/2023, 2:20 PMWell done @MarekM.... Impressive! Kerberos is never easy!
16 Views