This message was deleted.
# troubleshootings
Slackbot
03/09/2023, 4:36 AMThis message was deleted.
a
Amatya Avadhanula
03/09/2023, 4:39 AMDo you have streaming ingestion with late data or other batch ingestion with higher priority for this datasource + interval?
d
Dinesh
03/09/2023, 4:43 AMThe problem is with all data sources not with particular one
a
Amatya Avadhanula
03/09/2023, 4:45 AMCould you please share more info such as the task logs, overlord logs and maybe even the coordinator's logs related to compaction?
d
Dinesh
03/09/2023, 5:04 AMTASK
EROOR - " errorMsg": "org.apache.druid.segment.loading.SegmentLoadingException: Failed to load segment data_source_name_202..."
LOG -"Request failed with status code 404"
COORDINATOR LOGS
2023-03-08T133759,389 WARN [Coordinator-Exec--0] org.apache.druid.server.coordinator.duty.NewestSegmentFirstIterator - total segment size[1737266692] for datasource[data_source_name] and interval[2023-02-13T000000.000Z/2023-02-14T000000.000Z] is larger than inputSegmentSize[1073741824]. Continue to the next interval.
a
Amatya Avadhanula
03/09/2023, 5:06 AMCould you please check the segment availability as well?
Amatya Avadhanula
03/09/2023, 5:07 AMIt seems like the segment data might be missing from deep storage
Amatya Avadhanula
03/09/2023, 5:11 AM2023-03-08T133759,389 WARN [Coordinator-Exec--0] org.apache.druid.server.coordinator.duty.NewestSegmentFirstIterator - total segment size[1737266692] for datasource[data_source_name] and interval[2023-02-13T000000.000Z/2023-02-14T000000.000Z] is larger than inputSegmentSize[1073741824]. Continue to the next interval.This seems problematic as well. Could you please set the inputSegmentSize to a higher value and see if it helps? (Which version of Druid are you using?)
d
Dinesh
03/09/2023, 5:15 AMSegments are available for only this date ->2023-02-23
Dinesh
03/09/2023, 5:15 AMinputSegmentSize I didn't increased and tried.
As I am not familiar with the impact of doing so. Can you please tell me what it does if possible
a
Amatya Avadhanula
03/09/2023, 5:28 AMCould you please share which version you are using?
Amatya Avadhanula
03/09/2023, 5:28 AMI think inputSegmentSize has been deprecated
Amatya Avadhanula
03/09/2023, 5:30 AMFrom what I can see this was relevant when compaction ran on a single thread to limit the resources. However, compaction can run with several subtasks now and this config is no longer needed
d
Dinesh
03/09/2023, 5:37 AMThe version is 0.22.1
a
Amatya Avadhanula
03/09/2023, 5:40 AMYou could increase it to a large value and then increase maxNumConcurrentSubTasks as well
Amatya Avadhanula
03/09/2023, 5:40 AMHowever, I think the compaction failure is due to the segment data not being available
d
Dinesh
03/09/2023, 5:47 AMOk I will check
2 Views