We are running druid cluster using Docker on Azure...
# general
d
We are running druid cluster using Docker on Azure, and we're using Azure Blob Storage as the deep storage. We've noticed there is a
solr
process running in coordinator container. Does anyone know what is the role of
solr
process inside the Druid coordinator? It seems to be consuming quite a lot of CPU. Is there some druid configuration we can use to control this process and its cpu usage?
g
we don't use
solr
as part of Druid, is it possible this was added to your docker containers some other way?
d
I don't think so. It seems the parent process of
solr
is the coordinator process:
Copy code
PID    PPID COMMAND         COMMAND
1747821   10455 solr            ./solr
  10455   10401 java            /usr/bin/java --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -server -XX:+ExitOnOutOfMemoryError -XX:+UseG1GC -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.io.tmpdir=var/tmp -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dderby.stream.error.file=var/druid/derby.log -Xmx256m -Xms256m -XX:MaxNewSize=250m -XX:NewSize=250m -XX:MaxDirectMemorySize=1280m -Dlogfilename=coordinator -XX:+UseG1GC -XX:+ExitOnOutOfMemoryError -cp /tmp/conf/druid/cluster/_common:/tmp/conf/druid/cluster/master/coordinator-overlord:lib/*: org.apache.druid.cli.Main server coordinator
We noticed that
solr
is only present when we use azure for deep storage. Could it be related to
druid-azure-extensions
?
g
not that i can see; there is no reference to
solr
in our code (just searched the apache/druid repository). there is a
solr-solrj
library pulled as part of the extension
druid-ranger-security
but it just seems to be coming in as a dependency. i do not see code that actually launches a
solr
process
that's weird though
what container image are you using exactly? is this something reproducible?
d
We're using a private image off of the
apache/druid:25.0.0
image. Our private image has benign changes (adding config files), and runs
./druid.sh coordinator
at the end. We can reproduce it by using azure for deep storage.
g
very odd since the image doesn't have solr on it-- i tried
find / -name solr
if you have some steps to repro that would work outside your environment, we can try to reproduce it on our end