This message was deleted.
# troubleshooting
s
This message was deleted.
v
Do you have the
druid-kinesis-indexing-service
extension loaded everywhere?
j
we use kafka indexing service. it is loaded and unchanged from 0.23.0.
a
can you share full logs of your coordinator?
m
I also use kafka with 24 and have no problems. Make sure you have the extension loaded in the common config
Copy code
druid.extensions.loadList=[..."druid-kafka-indexing-service"...]
j
Here is the full log of coordinator:
@Michael Kutz We do have the kafka extensions loaded in common properties for all components.
g
Hmm I have a suspicion about what is going on here… @Jason Wang have you noticed if this resolved itself?
If my guess is right it should resolve after a background metadata store update completes
a
Coordinator's CompactSegments has
Copy code
Map<String, DataSourceCompactionConfig> compactionConfigs = compactionConfigList
            .stream()
            .collect(Collectors.toMap(DataSourceCompactionConfig::getDataSource, Function.identity()));
        final List<TaskStatusPlus> compactionTasks = filterNonCompactionTasks(indexingServiceClient.getActiveTasks());
indexingServiceClient.getActiveTasks() has not been changed as part of #12404
filterNonCompactionTasks
may think of tasks of an older version as compaction tasks
Copy code
private static List<TaskStatusPlus> filterNonCompactionTasks(List<TaskStatusPlus> taskStatuses)
  {
    return taskStatuses
        .stream()
        .filter(status -> {
          final String taskType = status.getType();
          // taskType can be null if middleManagers are running with an older version. Here, we consevatively regard
          // the tasks of the unknown taskType as the compactionTask. This is because it's important to not run
          // compactionTasks more than the configured limit at any time which might impact to the ingestion
          // performance.
          return taskType == null || COMPACTION_TASK_TYPE.equals(taskType);
        })
        .collect(Collectors.toList());
  }
g
Yeah I was thinking that's what's going on; filterNonCompsctionTasks returns the older tasks (since type is null) and then they can't be deserialized
Since the Coordinator deserializes them using ClientQueryTask which doesn't support kafka tasks
a
@Jason Wang Could you please share your overlord logs during this time as well?
j
Thanks folks for taking a look at this! It does seem to go away. I no longer see the error at least for the past few hours. Our overlord has warn log level and does not produce many logs. I will keep the thread posted if I see occurrences of this and will share the logs here. Thanks folks for taking a look at this.