Slackbot
09/23/2022, 4:57 PMMark Herrera
09/26/2022, 7:04 PMour requirement is respoonse time ie lsess then a minute with 2000 concurrent queries.
The error we are seeing is query time out becasue of mergebuffers and query time out.For what it's worth, I'm aware of one use case where the unwanted response time was caused by a datatype problem, i.e., the data was ingested as an int instead of a string.
Gian Merlino
09/27/2022, 3:58 AM+XX:+PrintGC to your jvm.config or by using jvm tools.) if any of these are not looking ideal, you may need to adjust settings.
2) ensure you don't have too many small segments: overheads per-segment can be high in this case. you can use the web console or sys.segments table to see the average number of rows per segment. Generally we target a few million rows per segment. Lower is OK if you have a smaller amount of data. But you don't want tons of segments that are, like, 10000 rows each
3) you can use a flame graph (https://support.imply.io/hc/en-us/articles/360033747953-Profiling-Druid-queries-using-flame-graphs) to see what specific code is taking up your processing time. very useful tool!Gian Merlino
09/27/2022, 4:01 AMintervals, and if you have a specific column you're often filtering on, consider applying secondary partitioning (aka clustering)
• Avoid unnecessary query time expressions
• Use the flame graph technique (see prior comment) to find where time is being spent