This message was deleted.
# troubleshooting
s
This message was deleted.
s
The newer segments are always loaded first.
1
m
An interesting thing I saw when I switched to http segment loading was that my streaming ingestion task was waiting to handoff segment for a really long time. This happen at the same time the cluster is loading many many segments for all the existing datasources. Since newer segments are always loaded first, I think what happened was that I was using the default value for
druid.coordinator.loadqueuepeon.http.batchSize
which is 1. This causes the coordinator load/drop cycle to take a really long time, preventing the newer segment from my realtime ingestion task being loaded. I increased
druid.coordinator.loadqueuepeon.http.batchSize
and no longer see this issue.
Also, reducing
maxSegmentsInNodeLoadingQueue
also help coordinator run to cycle faster as this config determines the max load/drop segment operations assigned to each Historical per cycle. Coordinator cycle faster help the handoff realtime segment gets assign faster