https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
a
Hi! A few questions to help debug this further • Is this occurring for a specific query only, (since you mentioned it runs in a second otherwise) or for all queries including simple ones which shouldn't timeout ever? • Is the url provided correctly pointing to druid? (For example is there something in logs to say that the query has hit the API on the broker and started successfully) Is there any exception or error in the logs that is causing it to the query to fail and timeout? The console also internally calls the same API, so this shouldn't happen.
a
1. Not to specific query -> example am using 20 sql queries means in random fresh some of the query response and some are time-out. Even in alternate response same query time-out which responds some time.
2. Yes it's pointing to Druid. Reason if always time-out / denied then this must be problem. But it works sometime and timeout sometime.
where the SQL query request logs will happen ?
in broker?
a
Yes, there should be some logs in the broker
v
if the api is posting multiple queries then this may be an issue with not having enough resources for concurrent queries.
How many queries per second are you posting?
In the console you are likely running one query at a time
a
in a api request I do call 1 query
In a page 1 do 10 queries with 10 api call separetly
I do*
but, not in single API call - multiple queries
here 1 point I want to mention, actually for 1 of the data source to kafka data stream going on with high volume (about 20 Lakhs) in parallel. Do this processor usage / memory usage... lead to delay the response for other datasource SQL queries ? Just asking. ?
v
yes. Depends on how many nodes you have and how many threads.
a
ok
nodes you mean here number of servers?
if yes, it's single server.
r
given a historical server, only one segment is processed at a time
if there's multiple queries in the queue and just one server, if all queries have the same priority (default), the queries will have random execution times, because it's not guarantee that the historical server will read all the segments for the first (incoming) query, it will try to balance the load between all the active queries, that can lead to very poor performance if you have a lot of queries and just one processing thread, for exmple
anothing thing, how are you encoding/escaping the query in the console versus in the application? if you are filtering by ID, try to make sure that it's using the index by using CAST(filtered_column AS varchar) = ? that can drastically improve response time, if you ingested as string but are querying as int, this will lead to a lot of unnecessary conversions and not using the right indexes
hopefully I think that in 26.0.0 (next version) with data catalog feat, Calcicte planning engine will get a refactor and may become smarter
k
One thing to note here - the Druid console uses limits and approximation by default, while an API call might not - do you get the same performance when you turn off the approximation and limit?
a
Thank You Renato, will check on your point. Thank you Kyle. Will try it.
Renato, am currently using 25.0.0 version.
Open in Slack
PreviousNext
Powered by