Are there any good tools that can analyze slow que...
# general
j
Are there any good tools that can analyze slow queries on a Druid cluster? We have enabled query logging on slow queries but there are gaps: 1. We need to log into Druid host to get the query logs 2. It's not easy to see which queries are the actual culprit since one bad query can slow everything down 3. There is no alerting capability
j
Hi JRob, there are a wealth of metrics that can be emitted: https://druid.apache.org/docs/latest/operations/metrics/ ... with this information and request logging it is fairly easy to locate problem queries.
g
IMO, the metrics are more useful than query logs for this, mainly because of this issue you raised: "one bad query can slow everything down"
I especially like using the metric
query/cpu/time
. It shows how much CPU time each query takes. You can break it down by query ID, query type, datasource, user, etc
j
I was thinking of something like a slow query analyzer. For example, what are the queries that are taking the most CPU time?
We have SQL queries enabled. I see a
sqlQuery/time
which is useful. Does
query/time
on the broker also include the SQL queries or is it purely Native queries?
j
Hi JRob, it includes SQL and Native both ... there are dimensions for each ... I believe the SQL Query ID dimension will be populated only if it is a SQL query.
👀 1
There is also
sqlQuery/planningTimeMs
, which I believe reports the SQL parse time.
j
Any ideas why dataSource would be null for some instances of
query/time
. Are these metadata queries?
Same question for
sqlQuery/time
j
Check other dimensions like Node Type, Service, Host, Query Type ... you can also group by SQL Query ID or Native Query ID and try to find the query in the Broker or Router logs. On a cluster I just checked I see NULL datasource only for queries going to the router.