Hello team, I am trying to bring up a pinot cluste...
# getting-startedm
Mugdha Goel
06/17/2022, 2:23 PMHello team, I am trying to bring up a pinot cluster on a gcp ubuntu 22.04 machine. I am able to bring up a docker container with all of pinot running but when I try to use the docker-compose here, I keep getting zookeeper cleanup exceptions.
Copy code
manual-pinot-server | 2022/06/17 13:46:34.432 INFO [StartServiceManagerCommand] [main] Started Pinot [SERVICE_MANAGER] instance [ServiceManager_12b62b9ab52c_-1] at 0.296s since launch
manual-pinot-server | 2022/06/17 13:46:34.434 INFO [StartServiceManagerCommand] [Start a Pinot [SERVER]] Starting a Pinot [SERVER] at 0.298s since launch
manual-pinot-controller | 2022/06/17 13:47:02.739 ERROR [StartServiceManagerCommand] [main] Failed to start a Pinot [CONTROLLER] at 30.356 since launch
manual-pinot-controller | java.lang.NullPointerException: null
manual-pinot-controller | at org.apache.helix.manager.zk.client.ZkConnectionManager.cleanupInactiveWatchers(ZkConnectionManager.java:112) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:95) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:91) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.zookeeper.ZkClient.connect(ZkClient.java:1620) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.zookeeper.ZkClient.<init>(ZkClient.java:186) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.ZkClient.<init>(ZkClient.java:87) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.client.ZkConnectionManager.<init>(ZkConnectionManager.java:41) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.client.SharedZkClientFactory.getOrCreateZkConnectionNamanger(SharedZkClientFactory.java:60) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.helix.manager.zk.client.SharedZkClientFactory.buildZkClient(SharedZkClientFactory.java:40) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.common.utils.ServiceStartableUtils.applyClusterConfig(ServiceStartableUtils.java:54) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.controller.BaseControllerStarter.init(BaseControllerStarter.java:162) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.service.PinotServiceManager.startController(PinotServiceManager.java:121) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:91) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.lambda$startBootstrapServices$0(StartServiceManagerCommand.java:249) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:302) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startBootstrapServices(StartServiceManagerCommand.java:248) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.execute(StartServiceManagerCommand.java:195) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.admin.command.StartControllerCommand.execute(StartControllerCommand.java:152) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.Command.call(Command.java:33) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.Command.call(Command.java:29) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at picocli.CommandLine.access$1300(CommandLine.java:145) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at picocli.CommandLine.execute(CommandLine.java:2078) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:161) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-controller | at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:192) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]Mugdha Goel
06/17/2022, 2:24 PM Copy code
manual-pinot-controller exited with code 0
manual-pinot-controller | SLF4J: Class path contains multiple SLF4J bindings.
manual-pinot-controller | SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-0.9.3-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
manual-pinot-controller | SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-0.9.3-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
manual-pinot-controller | SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-0.9.3-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
manual-pinot-controller | SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-0.9.3-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
manual-pinot-controller | SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-0.9.3-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
manual-pinot-controller | SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-0.9.3-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
manual-pinot-controller | SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
manual-pinot-controller | SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
manual-pinot-controller | WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
manual-pinot-controller | WARNING: An illegal reflective access operation has occurred
manual-pinot-controller | WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-0.9.3-jar-with-dependencies.jar) to method java.lang.Object.finalize()
manual-pinot-controller | WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
manual-pinot-controller | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
manual-pinot-controller | WARNING: All illegal access operations will be denied in a future release
manual-pinot-controller | 2022/06/17 13:47:04.217 INFO [StartControllerCommand] [main] Executing command: StartController -clusterName PinotCluster -controllerHost null -controllerPort 9000 -dataDir /tmp/data/PinotController -zkAddress manual-zookeeper:7000
manual-pinot-controller | 2022/06/17 13:47:04.226 INFO [StartServiceManagerCommand] [main] Executing command: StartServiceManager -clusterName PinotCluster -zkAddress manual-zookeeper:7000 -port -1 -bootstrapServices []
manual-pinot-controller | 2022/06/17 13:47:04.227 INFO [StartServiceManagerCommand] [main] Starting a Pinot [SERVICE_MANAGER] at 0.302s since launch
manual-pinot-controller | 2022/06/17 13:47:04.229 INFO [StartServiceManagerCommand] [main] Started Pinot [SERVICE_MANAGER] instance [ServiceManager_301ef6dd9be9_-1] at 0.305s since launch
manual-pinot-controller | 2022/06/17 13:47:04.229 INFO [StartServiceManagerCommand] [main] Starting a Pinot [CONTROLLER] at 0.305s since launch
manual-pinot-broker | 2022/06/17 13:47:04.474 ERROR [PinotServiceManager] [Start a Pinot [BROKER]] Failed to initialize Pinot Broker Starter
manual-pinot-broker | java.lang.NullPointerException: null
manual-pinot-broker | at org.apache.helix.manager.zk.client.ZkConnectionManager.cleanupInactiveWatchers(ZkConnectionManager.java:112) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:95) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:91) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.zookeeper.ZkClient.connect(ZkClient.java:1620) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.zookeeper.ZkClient.<init>(ZkClient.java:186) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.ZkClient.<init>(ZkClient.java:87) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.client.ZkConnectionManager.<init>(ZkConnectionManager.java:41) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.client.SharedZkClientFactory.getOrCreateZkConnectionNamanger(SharedZkClientFactory.java:60) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.helix.manager.zk.client.SharedZkClientFactory.buildZkClient(SharedZkClientFactory.java:40) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.pinot.common.utils.ServiceStartableUtils.applyClusterConfig(ServiceStartableUtils.java:54) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.pinot.broker.broker.helix.BaseBrokerStarter.init(BaseBrokerStarter.java:112) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.pinot.tools.service.PinotServiceManager.startBroker(PinotServiceManager.java:141) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:96) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:276) ~[pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:302) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:276) [pinot-all-0.9.3-jar-with-dependencies.jar:0.9.3-e23f213cf0d16b1e9e086174d734a4db868542cb]
manual-pinot-broker | 2022/06/17 13:47:04.485 ERROR [StartServiceManagerCommand] [Start a Pinot [BROKER]] Failed to start a Pinot [BROKER] at 30.351 since launchk
Kishore G
06/17/2022, 2:50 PM
Can you check the resources you have given to docker containers..
m
Mugdha Goel
06/17/2022, 4:00 PMwill reconfirm
Mugdha Goel
06/17/2022, 4:47 PM@Kishore G I confirmed that the docker configs look right