This message was deleted.
# troubleshootings
Slackbot
03/15/2023, 2:53 PMThis message was deleted.
i
Ian Roberts
03/15/2023, 2:56 PMlooks like it is trying to connect to localhost as the coordinator and not able to. Is the coordinator located on the same node? Does localhost resolve correctly?
a
ankit honeywell
03/15/2023, 2:59 PMcoordinator is on different server and the query server can reach it (verified by netcat)
i
Ian Roberts
03/15/2023, 3:06 PMWas this run previously on zookeeper as a single node cluster?
a
ankit honeywell
03/15/2023, 3:10 PMall the servers are single node including zookeeper. The behavior is similar for fresh setup or retires. Have also tried shutting down all servies and deleting the path in zookeeper and bringing up all services.
i
Ian Roberts
03/15/2023, 3:10 PMhmmm, it should be discovering the path via zookeeper. https://www.druidforum.org/t/router-is-failing-to-start-due-to-failure-to-connect-to-coordinator/8101
a
ankit honeywell
03/15/2023, 3:12 PMyeah I did follow this thread. Couldnt seem to be working for me.
i
Ian Roberts
03/15/2023, 3:13 PMcheck that the router can talk to zookeeper as well
Ian Roberts
03/15/2023, 3:14 PMalso that the router config has the correct
Copy code
druid.router.coordinatorServiceNamea
ankit honeywell
03/15/2023, 3:16 PMthere is connectivity between router and zookeeper. Also the router logs convey it.
ankit honeywell
03/15/2023, 3:22 PMservice name looks correct to me. I hope I am checking at correct location
i
Ian Roberts
03/15/2023, 3:26 PMare you running the coordinator under the overlord process?
Ian Roberts
03/15/2023, 3:27 PMI should say overlord under coordinator
a
ankit honeywell
03/15/2023, 3:29 PMI am not sure, I have not done any other changes explicitly, just kept it the default.
i
Ian Roberts
03/15/2023, 3:30 PMdo you have a separate process for overlord and for coordinator?
a
ankit honeywell
03/15/2023, 3:30 PMno, it is same
ankit honeywell
03/15/2023, 3:31 PMlogs file is a combined one too
i
Ian Roberts
03/15/2023, 3:32 PMthe issue is in the discovery process. The router cannot seem to find the coordinator. I would check the paths via the zkCli to make sure there are existing paths as it is configured. It may be because it is being run as one service
a
ankit honeywell
03/15/2023, 3:37 PMokay. when i query zk I can see the path for both coordinator and overload
ankit honeywell
03/15/2023, 3:42 PMI will try to run them separately and check the behaviour
i
Ian Roberts
03/15/2023, 3:43 PMcheck coordinator log as well to see if any errors stand out regarding zookeeper
a
ankit honeywell
03/15/2023, 3:46 PMI do see exceptions in master intermittently when connecting to data server for the ports 8083 and 8091
i
Ian Roberts
03/15/2023, 3:50 PMyour common.runtime.properties has druid.host=localhost. Try removing that
a
ankit honeywell
03/15/2023, 4:05 PMafter removing the services started fine except for router. but now all the services are showing this exception
i
Ian Roberts
03/15/2023, 4:07 PMI believe what is happening is each service is being registered as localhost. That is now in zookeeper. The default for druid.host is InetAddress.getLocalHost().getCanonicalHostName()
Ian Roberts
03/15/2023, 4:08 PMplease try that and try cleaning out the zk directories as you did before
a
ankit honeywell
03/15/2023, 4:10 PMokay, what you are suggesting is stopping all services, cleaning zk and starting them back (no changes in common.runtime.properties)
i
Ian Roberts
03/15/2023, 4:11 PMyou need to put in druid.host with the default value, localhost is for single node clusters
Ian Roberts
03/15/2023, 4:12 PMdruid.host=InetAddress.getLocalHost().getCanonicalHostName()
a
ankit honeywell
03/15/2023, 4:12 PMokay let me do that
i
Ian Roberts
03/15/2023, 4:17 PMyou should just be able to comment out druid.host on all common.runtime.properties and it will use the default value
Ian Roberts
03/15/2023, 4:18 PMdid you previously just remove the value?
Ian Roberts
03/15/2023, 4:18 PMand that is why it was throwing null
a
ankit honeywell
03/15/2023, 4:20 PMprevious i removed the entire line
i
Ian Roberts
03/15/2023, 4:21 PMhmm ok
a
ankit honeywell
03/15/2023, 4:28 PM😕
i
Ian Roberts
03/15/2023, 4:30 PMI am unsure why it is not working. It should have used the default when you removed from all common.runtime.properties files and cleaned out zookeeper
Ian Roberts
03/15/2023, 4:30 PMthat is definitely the issue though
Ian Roberts
03/15/2023, 4:31 PMyou could put the ip or hostname in on each one but I do not believe that is necessary
a
ankit honeywell
03/15/2023, 4:33 PMyou mean druid.host to their respective ips?
i
Ian Roberts
03/15/2023, 4:40 PMyes
a
ankit honeywell
03/15/2023, 5:12 PMsetting them to their respective ips all services are up and running
ankit honeywell
03/15/2023, 5:12 PMno errors so far
🎉 1
ankit honeywell
03/15/2023, 5:32 PMdruid cluster is up and running
ankit honeywell
03/15/2023, 5:33 PM@Ian Roberts Thank you for your time and expertise. I could not have made it work otherwise 👏
i
Ian Roberts
03/15/2023, 5:37 PMyou are very welcome, glad it is working
🙌 1