Hi people, I am facing an issue with starting ThirdEye on top of Pinot. I have got pinot successfully set up and running. Now, I am trying to run ThirdEye on top of this pinot using the docker apachepinot/thirdeye image. After running the following docker command, an Error stating

Database may be already in use

Please find the attached log file. Any help is appreciated!

docker run \
    --network=pinot-demo \
    --name thirdeye \
    -p 1426:1426 \
    -p 1427:1427 \
    -d apachepinot/thirdeye:latest

Slack Conversation

how do you run pinot in docker?

I start the pinot for my own data and cluster configs. Like this,

# zookeeper

docker run \
    --network=pinot-demo \
    --name  pinot-zookeeper \
    --restart always \
    -p 2181:2181 \
    -d zookeeper:latest

# controller

docker run -ti \
    --network=pinot-demo \
    --name pinot-controller \
    -p 9000:9000 \
    -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms1G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-controller.log" \
    -d apachepinot/pinot:latest StartController \
    -zkAddress pinot-zookeeper:2181

# broker

docker run -ti \
    --network=pinot-demo \
    --name pinot-broker \
    -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-broker.log" \
    -d apachepinot/pinot:latest StartBroker \
    -zkAddress pinot-zookeeper:2181

# server

docker run -ti \
    --network=pinot-demo \
    --name pinot-server \
    -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-server.log" \
    -d apachepinot/pinot:latest StartServer \
    -zkAddress pinot-zookeeper:2181

@ayush sharma You can use helm to fire up everything in one shot

@Suvodeep Pyne You mean, running pinot using helm instead of docker might fix the docker start of ThirdEye ? I'll try that and let you know.

@ayush sharma For thirdeye, should be something simliar, Check out the

kubernetes

directory inside

incubator-pinot

./helmw install thirdeye .

Hi @Suvodeep Pyne , Thank you for the helm & kubernetes guidance. I have now set up a pinot cluster on minikube using helm and have created an offline table (not pushed the data yet.) Now, when I try to helm install thirdeye using

helm install thirdeye ~/incubator-pinot/kubernetes/helm/thirdeye -n my-pinot-kube

it just throws an error

Error: found in Chart.yaml, but missing in charts/ directory: mysql

I think, this could be a silly mistake of mine. Could you please help me out?

Hi @ayush sharma can you take a look at the ./install.sh or helw script?

here is the ./install.sh of thirdeye

#!/usr/bin/env bash

base_dir=$(dirname "$0")
cd ${base_dir};

# fetch dependencies. example: mysql. See Chart.yaml
helm dependency update

# Note:
# - initdb files must end with .sql
# - When injecting yaml config via terminal, the period ('.') must be escaped and quoted
helm install thirdeye . \
  --set-file mysql.initializationFiles."initdb\.sql"="./config/initdb.sql" \
  --set-file thirdeye.config.dataSources="./config/data-sources/data-sources-config.yml" \
  $@

@ayush sharma I think you can just run

./install.sh -n my-pinot-kube

to install thirdeye.

It worked like a charm! Thank you @Suvodeep Pyne

🎉