This message was deleted.

A query timeout could occur for multiple reasons. If this is occurring when segment publish and handoff happen on a streaming ingestion, is it possible that the resources on the streaming Peons are too tight? Are there enough http threads ? I am assuming by "druid pushes the data to Historical nodes" you mean you are running a streaming ingestion and this occurs at publish and handoff of new segments.

Yes, We are running streaming job and observing this behaviour in Peak traffic hours. We have seen that the cpu and memory utilisation was optimal. However whenever handoff happens Druid utilises all the 20 Peons.

Yes. A streaming ingestion will perform a checkpoint at the duration of the tasks, at that point all reading tasks go into a publishing state and a new set of tasks is started to continue consuming. So you need 2x the task slots while publishing.

Exactly which variable you are referring here?

At

taskDuration

all reading tasks will transition to the publishing state and another

taskCount

set of tasks will be created to continue consuming. The publishing tasks will continue to respond to queries until a Historical announces that it has the segment(s), at that point the task completes and releases the worker slot (MM Peons available

druid.worker.capacity

). There is ongoing work to make these transitions less "needy". For example, there is work being done to cascade the checkpoint such that you don't need to transition all tasks at the same time, but this is still future. So for now, you will need 2 x

taskCount

worker slots to deal with the checkpoint.