https://pinot.apache.org/ logo
Join Slack
Powered by
Hello, I'm using the following tutorial to use S3 ...
# troubleshooting
k
Hello, I'm using the following tutorial to use S3 and Pinot in Docker (https://docs.pinot.apache.org/users/tutorials/use-s3-and-pinot-in-docker) but when I try to start up the controller using the the docker command provided in the tutorial I keep on getting below error:
Copy code
Executing command: StartController -configFileName /tmp/controller.conf
Caught exception while starting controller, exiting.
java.lang.RuntimeException: Error: Unable to find controller config file /tmp/controller.conf
I've created the controller.conf file and placed it in the /tmp/pinot-s3-docker folder on my EC2 instance. Any help would be greatly appreciated.
m
It is looking for the 
controller.conf
in 
/tmp
folder, but you put it in 
/tmp/pinot-s3-docker
?
x
From where you run the docker run command, did you mount your local directory to the docker image?
You can use -v to do so
k
I used this command (edited to pass in my aws keys) :
Copy code
docker run --rm -ti \
    --name pinot-controller \
    --network=pinot-demo \
    -p 9000:9000 \
    --env AWS_ACCESS_KEY_ID=<aws-access-key-id> \
    --env AWS_SECRET_ACCESS_KEY=<aws-secret-access-key> \
    --mount type=bind,source=/tmp/pinot-s3-docker,target=/tmp \
    apachepinot/pinot:0.6.0-SNAPSHOT-ca8545b29-20201105-jdk11 StartController \
    -configFileName /tmp/controller.conf
I tried -v as well but keep getting the error. I used this command
Copy code
docker run --rm -ti \
    --network=pinot-demo \
    --name pinot-controller \
    -p 9000:9000 \
    -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms1G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-controller.log" \
    -d ${PINOT_IMAGE} StartController \
    -zkAddress pinot-zookeeper:2181
with the mount and ran docker inspect on the pinot-controller and it shows the mount.
Copy code
"Mounts": [
            {
                "Type": "volume",
                "Name": "8c86a92945bf6c16083a60e373bc80112afa9fac7360b0c1394dd90599b9dd5e",
                "Source": "/var/snap/docker/common/var-lib-docker/volumes/8c86a92945bf6c16083a60e373bc80112afa9fac7360b0c1394dd90599b9dd5e/_data",
                "Destination": "/opt/pinot/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/tmp/pinot-s3-docker",
                "Destination": "/tmp",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "volume",
                "Name": "66cba96eb1246f586b9864227cd3498fde4f6dbdd9ae422721f84ecb59c6c212",
                "Source": "/var/snap/docker/common/var-lib-docker/volumes/66cba96eb1246f586b9864227cd3498fde4f6dbdd9ae422721f84ecb59c6c212/_data",
                "Destination": "/opt/pinot/configs",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ]
x
I think this command is good
Can you check if the directory is there with the files
Or you can just run the bash cmd to enter into the image/container
k
I checked the directory on the pinot-controller container and the file is not there.
x
Hmm, then I feel the mounting doesn’t work
Do you have right credential
k
Yeah, can't figure out what the issue is. I'm running commands as root
x
Hmm, can you try to mount it to a different sub directory ?
Like /tmp/pinot
k
So I mounted to tmp/pinot and the container now has that sub directory but no controller.conf file there. I used vim to create the file on the host, could that be causing any issues?
Not sure how but the directory, pinot-s3-docker ended up in tmp/snap.docker/tmp/pinot-s3-docker. When I ran the docker run it started working but then I had to make change to a typo in controller file and when I pushed the change, it wouldn't update the file in the snap directory. Now I'm back to where I started 😕
Resolved. I had to uninstall and reinstall docker. Docker was installed as an Ubuntu Snap (https://stackoverflow.com/questions/63287386/docker-wont-copy-files-from-the-container-to-the-hosts-tmp-folder). Was able to start the controller.
m
Thanks @Kamal Chavda. Perhaps you may consider adding this to the FAQ in pinot docs? Details in #C023BNDT0N8.
k
Will do Mayank. Thanks again for your help guys!
👍 1
Open in Slack
PreviousNext
Powered by