I am now having issues setting up Kafka for Datahub. I am using AWS MSK with SASL/SCRAM. After doing some searching I see that I need to make the following update to my

values.yaml

file:

global:
  springKafkaConfigurationOverrides:
    security.protocol: SASL_SSL
    sasl.mechanism: SCRAM-SHA-512
    sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="username" password="password":

However, with only these changes the

datahub-kafka-setup-job

fails, and I see the following in the logs.

org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
        at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:535)
        at org.apache.kafka.clients.admin.Admin.create(Admin.java:75)
        at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49)
        at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:138)
        at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
        at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
        at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
        at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:82)
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:134)
        at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:73)
        at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
        at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:508)
        ... 4 more

Does anyone know if there is something that I am missing or did incorrectly?

The trailing colon should be a semi-colon, see if that fixes it 🙂