Hi, I am running into an error when submitting supervisor in druid I have a realtime pipeline running using EMR (Apache Flink) and MSK in AWS . The job reads from Apache Kafka topic, processes it, and then writes to the output Kafka topic. Once data lands in Kafka topic (MSK), the data is written to Druid segment through a supervisor created in Druid However, when I try to

Submit supervisor

using

Load Data

option in Druid console, I get an error on the supervisor created

{
  "dataSource": "master_order_event",
  "stream": "realtime-pipelines-pos-db.pos_local.orders",
  "partitions": 0,
  "replicas": 1,
  "durationSeconds": 3600,
  "activeTasks": [],
  "publishingTasks": [],
  "minimumLag": {},
  "aggregateLag": 0,
  "suspended": true,
  "healthy": true,
  "state": "PENDING",
  "detailedState": "PENDING",
  "recentErrors": [
    {
      "timestamp": "2026-03-06T06:35:28.082Z",
      "exceptionClass": "org.apache.kafka.common.config.ConfigException",
      "message": "Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found.",
      "streamException": false
    },
    {
      "timestamp": "2026-03-06T06:36:37.461Z",
      "exceptionClass": "org.apache.kafka.common.config.ConfigException",
      "message": "Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found.",
      "streamException": false
    },
    {
      "timestamp": "2026-03-06T06:37:49.854Z",
      "exceptionClass": "org.apache.kafka.common.config.ConfigException",
      "message": "Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found.",
      "streamException": false
    }
  ]
}

I have tried to troubleshoot this issue with a few steps: I added the

aws-msk-iam-auth-2.2.0-all.jar

to

/opt/druid/extensions/druid-kafka-indexing-service/aws-msk-iam-auth-2.2.0-all.jar

1. Placed aws-msk-iam-auth-2.2.0-all.jar in /opt/druid/extensions/druid-kafka-indexing-service/ Druid Middle Manager docker container inside EC2 instance 2. Restarted Druid Middle Manager docker container After this, I could see the jars are in correct place, the druid-indexing-service is added to common-runtime.properties, the container was restarted but still running into Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found. Then I copied the jar to another path

cp /opt/druid/extensions/druid-kafka-indexing-service/aws-msk-iam-auth-2.2.0-all.jar /opt/druid/lib/

Is this the right solution? I am still running into issues

I had a similar issue in the past while I was trying to load

druid-protobuf-extensions

. Here's my advice: 1. The proper place for your extensions looks to be

opt/druid/extensions/

2. Ensure that

druid-kafka-indexing-service

appears in your

druid.extensions.loadList

3. Ensure that

druid.extensions.loadList

is not overridden in other properties files (i.e.

_common/common.runtime.properties

is loaded first but

middleManager/runtime.properties

will override it) 4. Ensure you rollout the

common.runtime.properties

to all nodes in the cluster. You may find that the Coordinator/Overlord also requires the extension to be loaded. 5. Ensure you restart the service after making any config change or changing any of the jar files. Good luck!