Hey All, By any chance is there any limitation tha...
# generalp
pratush maheshwari
03/12/2024, 11:48 AMHey All, By any chance is there any limitation that number only a certain number of requests can be hit at the same time in druid?
k
Kyle Hoondert
03/12/2024, 12:41 PMThere could be a limited number of concurrent connections depending on your config - have a look at https://druid.apache.org/docs/latest/configuration/#server-configuration
p
pratush maheshwari
03/12/2024, 12:55 PMI don’t see anything given like this in config, also there is no default value given in docs, so what can it be?
pratush maheshwari
03/12/2024, 12:56 PMSince I have faced intermittent challenges while retreating so trying to figure out can it be due to number of requests hitting druid at the same time
k
Kyle Hoondert
03/12/2024, 1:02 PMSo the default is
Copy code
max(10, (Number of cores * 17) / 16 + 2) + 30p
pratush maheshwari
03/13/2024, 8:22 AMI am getting 500 Internal server error, anyway to track or debug this?
There are multiple queries hitting at the same time, so I am suspecting it may be due to that.
However while I tried it using thread pool executor I am not able to see this
k
Kyle Hoondert
03/13/2024, 9:20 AMoh - a 500 is different - see if you can see anything in your router or broker logs
p
pratush maheshwari
03/13/2024, 11:31 AMCan you help me to understand the parameter -
enableParallelMerge
pratush maheshwari
03/14/2024, 6:34 AMAnyone?
l
Laksh Singla
03/14/2024, 7:07 AMp
pratush maheshwari
03/14/2024, 7:11 AMSo my doubt here is, if I run the 2-3 different queries at the same time, is there a possibility that results getting mixed with this option enabled?
l
Laksh Singla
03/14/2024, 7:17 AMnope
Laksh Singla
03/14/2024, 7:18 AMis there any inclination towards parallel merge being the cause of 500?
p
pratush maheshwari
03/14/2024, 7:19 AMYes, We are trying to hit two different API calls at the same time but receiving the response of one in another
pratush maheshwari
03/14/2024, 7:20 AMAlthough druid from its own giving the response, but in the way it is expecyed
l
Laksh Singla
03/14/2024, 7:21 AMDruid merges the results in multiple places, therefore I was curious about the parallel merge being a suspect here.
p
pratush maheshwari
03/14/2024, 7:21 AMAlso a couple of times I saw connection has been shut down, any log where I can see the complete log?
pratush maheshwari
03/14/2024, 7:21 AMI tried checking broker log, couldn’t find anything useful
l
Laksh Singla
03/14/2024, 7:22 AMyou can enable debug logging, though I suspect that would be too verbose
p
pratush maheshwari
03/14/2024, 7:22 AMYes only thing that made me asked this question was enableParallelMerge parameter
l
Laksh Singla
03/14/2024, 7:23 AMidt parallel merge should be the culprit. You can disable it from the config and check the results
p
pratush maheshwari
03/14/2024, 7:24 AMSure
pratush maheshwari
03/14/2024, 7:24 AMHow do we enable log debugging?
pratush maheshwari
03/14/2024, 7:24 AMI would like to try that too