This message was deleted.
# generals
Slackbot
09/15/2023, 7:31 AMThis message was deleted.
b
Bibek Sahoo
09/15/2023, 7:37 AMYou can increase
druid.worker.capacityBibek Sahoo
09/15/2023, 7:37 AMp
Priyanka Khokrale
09/15/2023, 7:40 AMAfter adding this property it says 'middleManger configs (druid.indexer.runner.javaOptsArray or druid.worker.capacity) is present in middleManager/runtime.properties, add jvm.config for all other services'
b
Bibek Sahoo
09/15/2023, 7:41 AMTo increase the slots, You need to update
druid.worker.capacity=new valueruntime.propertiesBibek Sahoo
09/15/2023, 7:42 AMgenerally i have seen 1/3rd or 1/2 for number of cores is used
Bibek Sahoo
09/15/2023, 7:42 AMFirst check if the property is present in runtime.properties, if yes then update else add new
p
Priyanka Khokrale
09/15/2023, 7:45 AMYes i have added the same in runtime.properties. So i have set it 'druid.worker.capacity=4' , then still the same error..and i have only 1 runtime.properties file
b
Bibek Sahoo
09/15/2023, 7:46 AMShare the config and the error log (may be log snippet from start failure )
p
Priyanka Khokrale
09/15/2023, 7:48 AMThis is the runtime.properties file from middle manager:
druid.service=druid/middleManager
druid.plaintextPort=8091
druid.worker.capacity=4
druid.worker.baseTaskDirs=[\"var/druid/task\"]
druid.indexer.task.hadoopWorkingPath=var/druid/hadoop-tmp
b
Bibek Sahoo
09/15/2023, 7:51 AMIs this the complete file 😞
Bibek Sahoo
09/15/2023, 7:51 AMCould you attach the log here as well please
p
Priyanka Khokrale
09/15/2023, 7:52 AMYes sending
Priyanka Khokrale
09/15/2023, 7:52 AMAs i am very new to this i really dont have much knowledge
b
Bibek Sahoo
09/15/2023, 7:53 AMThats ok, no worries, Just share the complete runtime.properties for the middle manager and the log and let me check, if i can find anything
p
Priyanka Khokrale
09/15/2023, 7:53 AMmiddleManager.stdout.log
Priyanka Khokrale
09/15/2023, 7:55 AMI have also made some changes in indexer, runtime.properties
Priyanka Khokrale
09/15/2023, 7:56 AMThanks a lot for helping me
b
Bibek Sahoo
09/15/2023, 8:01 AMI do not see the error that you were saying first file is empty and there the worker.capacity is set to 2 only.
Could you please restart the middle manager and share the latest log
p
Priyanka Khokrale
09/15/2023, 8:02 AMokay wil restart
b
Bibek Sahoo
09/15/2023, 8:04 AMYou mentioned somewhere you updated indexer, are you using middle manager/indexer?
p
Priyanka Khokrale
09/15/2023, 8:05 AMis it like we can use only one?
b
Bibek Sahoo
09/15/2023, 8:06 AMSeems like
Bibek Sahoo
09/15/2023, 8:06 AMThe Indexer is an optional and experimental feature. Its memory management system is still under development and will be significantly enhanced in later releases.
The Apache Druid Indexer process is an alternative to the MiddleManager + Peon task execution system. Instead of forking a separate JVM process per-task, the Indexer runs tasks as separate threads within a single JVM process.
The Indexer is designed to be easier to configure and deploy compared to the MiddleManager + Peon system and to better enable resource sharing across tasks.
Bibek Sahoo
09/15/2023, 8:07 AMand for indexer :
# Number of tasks (druid.worker.capacity) is automatically
# determined based on available processor.
Bibek Sahoo
09/15/2023, 8:07 AMIf you go to your console do you see middle managers running in the services tab?
Bibek Sahoo
09/15/2023, 8:09 AMIf you are using indexer, then increasing the processor for the data node should be the solution i believe.
p
Priyanka Khokrale
09/15/2023, 8:11 AMyes middle manager was runnning
b
Bibek Sahoo
09/15/2023, 8:12 AMShould we do a quick zoom session
p
Priyanka Khokrale
09/15/2023, 8:12 AMcan we do google meet, i will send the link
b
Bibek Sahoo
09/15/2023, 8:12 AMsure
p
Priyanka Khokrale
09/15/2023, 8:13 AMyes just a minute
Priyanka Khokrale
09/15/2023, 8:15 AMPriyanka Khokrale
09/15/2023, 8:16 AMplease join
b
Bibek Sahoo
09/15/2023, 9:08 AMThank you for the call, We have done the changes to the config and the slots has been increased but as we discussed the processor xmx/xms/direct memory will have to be changed as per the resources available to run more jobs.
p
Priyanka Khokrale
09/15/2023, 9:27 AMThank you so much for resolving an issue..Now its working properly. 😊
👍 1