https://pinot.apache.org/ logo
Join Slack
Powered by
Hi everyone, I would like to spin up the most basi...
# troubleshooting
s
Hi everyone, I would like to spin up the most basic, lightweight running pinot cluster to use for things like testing in CI. I was trying to run it in a single docker container where I can control the mem/cpu usage. I came across 
pinot-admin.sh StartServiceManager
but have been unsuccessful in getting this up and running and the errors are not helpful. I know I need zookeeper running ahead of time. Can someone just provide a way to do this? It must be a common thing and yet I cannot find anything which brings up the controller, broker and server in the docs. I would expect something like 
pinot-admin.sh StartServiceManager -zkAddress localhost:2181 -clusterName PinotCluster -bootstrapServices CONTROLLER BROKER SERVER
to just work, perhaps I am missing something.
For more, information, I am running pinot in docker. Here is how I envisioned doing this:
Copy code
docker run --entrypoint='' apachepinot/pinot:0.11.0 bash -c "bin/pinot-admin.sh StartZookeeper & sleep 10 && bin/pinot-admin.sh StartServiceManager"
which brings up zookeeper in a separate process, and after sleeping for a bit brings up the controller, broker and server. There should be nothing special about the shell logic, I encounter the same problem running the commands directly in the container, but that should be a quick reproducible way to test this.
Also I know I can run each pinot component separately (e.g. 
StartController
) and this does work, and also run this with docker compose. But I was hoping the existence of 
StartServiceManager
would make this simpler for testing in CI.
r
you can see how https://github.com/python-pinot-dbapi/pinot-dbapi/tree/master/.github uses the docker image to test python DBAPIs
s
Thanks for the reference. However this uses QuickStart. I am looking for 
StartServiceManager
and run this with our own tables and data.
r
correct. you can use the generic EmpytQuickstart to bring up the cluster and add further commands to load the tables right?
s
Wow this is wild 😂 
QuickStart -type empty
. Where is this documented, besides the blog?
r
🙂 please feel free to submit a documentaiton improvement PR.
s
Awesome, so that seems to be a workaround. However it does bring up a minion which I don’t want, as I want to keep this as light as possible.
r
subsequence args after the 
-type EMPTY
is channel through the empty quickstart command. so it should be able to take some customization. not sure if not-starting minion is an option
s
It would be great to get an answer on how to get 
StartServiceManager
working, still. I am not sure why my approach was not working, so let’s keep this thread running.. hopefully someone will chime in.
r
i dont think it has to do with the service manager but more likely the way you start zk and the service manager together was not correct. but the quickstart is there for specific this reason.
and for minions, yeah i see it is hard-coded to start one, it would be great if we allow config override using cli arguments. we are happy to accept contributions to this
s
Yeah I thought as much, which is why I provided my start up command hoping someone could easily spot the issue. No luck so far 😞
r
from a first look i dont think it is specifically for that command. see how the quickstart integration test script on pinot does the same. what I felt like it is the combination of the command. I would recommend create a github issue to allow emptyquickstart to accept component-wise configuration and start those when configuration is provided.
Open in Slack
PreviousNext
Powered by