This message was deleted.
# troubleshootings
Slackbot
03/20/2023, 2:01 PMThis message was deleted.
v
Vijay Narayanan
03/20/2023, 2:07 PMWhat is the advertised listener in kafka?
Vijay Narayanan
03/20/2023, 2:09 PMAnd what is the kafka server url you are specifying in druid. I am wondering whether you have a mismatch here. Sometimes when you specify bootstrap server ip but kafka advertised listener has host name the kafka will return the host name and that may be a problem.
m
Martin Maqueira
03/20/2023, 2:25 PM@Vijay Narayanan Yes i know what you are talking about. I deal with it before. It's still being a problem for scaling horizontally. I have to manually map each name of any kafka instance (pod) as a External Service on K8S . I couldn't find a solution for it
Martin Maqueira
03/20/2023, 2:35 PMAll task are ending like this
Martin Maqueira
03/20/2023, 2:35 PM2023-03-20T142714,932 INFO [[index_kafka_qps_b0d6d240ef9db80_pikaobae]-publish] org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver - Published [1] segments with commit metadata [{nextPartitions=SeekableStreamStartSequenceNumbers{stream='qps', partitionSequenceNumberMap={0=432062}, exclusivePartitions=[]}, publishPartitions=SeekableStreamEndSequenceNumbers{stream='qps', partitionSequenceNumberMap={0=432062}}}]
2023-03-20T142714,933 INFO [[index_kafka_qps_b0d6d240ef9db80_pikaobae]-publish] org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver - Published segments: [qps_2023-03-01T000000.000Z_2023-04-01T000000.000Z_2023-03-01T000003.462Z_5100]
2023-03-20T142714,935 INFO [[index_kafka_qps_b0d6d240ef9db80_pikaobae]-publish] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Published 1 segments for sequence [index_kafka_qps_b0d6d240ef9db80_0] with metadata [AppenderatorDriverMetadata{segments={index_kafka_qps_b0d6d240ef9db80_0=[SegmentWithState{segmentIdentifier=qps_2023-03-01T000000.000Z_2023-04-01T000000.000Z_2023-03-01T000003.462Z_5100, state=APPENDING}]}, lastSegmentIds={index_kafka_qps_b0d6d240ef9db80_0=qps_2023-03-01T000000.000Z_2023-04-01T000000.000Z_2023-03-01T000003.462Z_5100}, callerMetadata={nextPartitions=SeekableStreamStartSequenceNumbers{stream='qps', partitionSequenceNumberMap={0=432062}, exclusivePartitions=[]}, publishPartitions=SeekableStreamEndSequenceNumbers{stream='qps', partitionSequenceNumberMap={0=432062}}}}].
2023-03-20T142714,936 INFO [[index_kafka_qps_b0d6d240ef9db80_pikaobae]-publish] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Published segments: [qps_2023-03-01T000000.000Z_2023-04-01T000000.000Z_2023-03-01T000003.462Z_5100]
2023-03-20T142714,937 INFO [[index_kafka_qps_b0d6d240ef9db80_pikaobae]-publish] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Saved sequence metadata to disk: []
2023-03-20T142812,551 WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [1] Segments
2023-03-20T142912,552 WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [1] Segments
2023-03-20T143012,475 WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [1] Segments
2023-03-20T143112,552 WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [1] Segments
2023-03-20T143212,482 WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [1] Segments
2023-03-20T143312,550 WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [1] Segments
2023-03-20T143412,475 WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [1] Segments
v
Vijay Narayanan
03/20/2023, 3:46 PMIt seems like task is ingesting but is not able to commit offset and handoff. Are segments written to s3 ?