This message was deleted.

depending on the query you could break it up into multiple queries and fetch say 500 rows each in 6 queries and thus get the 3000 in parallel. Can you paste your query here?

Thanks for your response. but sometimes the same query returns the result with 1 second. I don't know why it taking more than 1 minute. here is the query I am executing:- ========================= data = druid_client.scan( datasource = <table name>, granularity = 'none', intervals = f"{str(start_time)}/{str(end_time)}", context={"timeout": 1000}, filter = Dimension('route_id') == route_id)

do you have historical or broker cache enabled?

I think cache is not enabled? But I have asked platform team to confirm it. Only one roiute_id using in the query.

cache being enabled will cause variation in query times. Beyond that you need to look at how many segments are being scanned by the query and the size of segments to see where the bottle neck is (also are other queries running at the time?).

There is only one segment available. As I have set segment_granularity = Week and Task Duration = PT164H

how big is this segment?

segment size I defined as 300-400 MB only...I believe there could be some issue related to configuration. I was checking the druid server CPU/Memoy usage during data fetching. I found that coordinator's cpu usage is high at that time. So it might be impacting us.

Only one segment, and scanning takes over a minute? I wonder why the coordinator's cpu is high. What do you have for start_time/end_time? Is the query better for smaller time intervals?

Caching is enabled at broker and historical level, but still taking longer time to fetch the data. Start_time = timestamp(date+6:00AM) end_time = timestamp(date+7:00PM) Between this time interval, i want to fetch the data from druid. As I hit the query, the broker might be adding my request to the queue.I'm simply speculating.

Yes, could be that the broker is overwhelmed (or any other node I guess), and needs tuning for your load.