This message was deleted.
# general
s
This message was deleted.
b
1. This will probably not help much, as Druid will not use multiple replicas simultaneously in a query. 2. This will help if you're submitting the same query multiple times. If you have different queries or realtime data then it will not help. 3. Increasing threads will not help if you're already seeing high CPU. a. Decreasing the timeout can help, this can be done by setting the timeout property in the query context, or globally by setting the property in the broker runtime properties. I have found the best tuning approach is to first look at the queries (joins, aggregations, approximations) and data (segment sizes, compaction, partitioning) for efficiency improvements before trying to adjust cluster settings.
j
Thanks for taking the time to respond to my question. It means a lot to me! The aim to tune is to decrease query timeout. Most of the queries are aggregation, single query is lightful and responds in 2s. But the concurrent queries up to 500~1k in 10s. When the historical cpu high, it shows there has scan segment pending and leads query timeout… We has 10 historical nodes, the historical http numThread is 60, processing threadNum is 30.
b
How many cores do the historical servers have? How many brokers do you have, and how many cores do they have? How are the queries being generated? Do you have an application querying Druid?
The easiest way to adjust the timeout is to set the timeout in the context. https://druid.apache.org/docs/latest/querying/query-context#general-parameters If that isn't possible, then you can set
druid.server.http.defaultQueryTimeout
in the broker properties to set the default timeout for all queries that do not have the timeout set in the query context.
j
historical has 32 cores, we have 2 brokers and broker has 16 cores. We don’t enable broker cache, so brokers seems don’t have too much pressure. If the current cluster size cannot handle a large number of queries, even if I decrease the timeout threshold, there will still be a large number of query timeouts. Now our maxQueryTimeout is 60s.
m
I think I remember that you could set maxQueryTimeout to 180s as the max. That said, I would recommend focusing on the query, @Jiaojiao Fu It's fairly eye-opening how tweaking the components of your Druid query can make a big difference.
b
There isn't a limit to what you can set maxQueryTimeout - but there are practical limits and after about 10 minutes you might start seeing other timeouts such as ELB.
m
Good to know @Benjamin Hopp I think our Druid admins set the 3 minute limit. The standard timeout was 1 minute and I sometimes tweaked the limit to see if a query might run if it was given just a bit more time. Many customers, I would think, cache the query and results to avoid hitting Druid except for new queries, that's what we did...