Hi Team, I am trying to use hdfs as deepatorage an...
# general
r
Hi Team, I am trying to use hdfs as deepatorage and created controller.conf server.conf and broker.conf files. Kindly review and Provide your valuable suggestions on attached file. Please suggest/guide if anything needs to change or add.
a
@User you have added script in wrong file. e.g: For Broker create a file broker.conf and add following in that file.
Copy code
pinot.set.instance.id.to.hostname=true
pinot.server.grpc.enable=true
After this, run the following in your terminal ( or you have also create a shell script file) to start the broker
Copy code
exportHADOOP_HOME=/user/local/hadoop
exportHADOOP_VERSION=2.7.1
exportHADOOP_GUAVA_VERSION=11.0.2
exportHADOOP_GSON_VERSION=2.2.4
exportGC_LOG_LOCATION=/tmp/gc_log/
exportPINOT_VERSION=0.7.1
exportPINOT_DISTRIBUTION_DIR=/tmp/my_data_2/apache-pinot-incubating-0.7.1-bin
exportSERVER_CONF_DIR=/tmp/my_data_2/apache-pinot-incubating-0.7.1-bin/bin
exportZOOKEEPER_ADDRESS=00.000.00.000:2181
export CLASSPATH_PREFIX="${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-
${HADOOP_VERSION}.jar:${HADOOP_HOME}/share/hadoop/common/lib/hadoop-annotations-
${HADOOP_VERSION}.jar:${HADOOP_HOME}/share/hadoop/common/lib/hadoop-auth-
${HADOOP_VERSION}.jar:${HADOOP_HOME}/share/hadoop/common/hadoop-common-
${HADOOP_VERSION}.jar:${HADOOP_HOME}/share/hadoop/common/lib/guava-
${HADOOP_GUAVA_VERSION}.jar:${HADOOP_HOME}/share/hadoop/common/lib/gson-
${HADOOP_GSON_VERSION}.jar"
export JAVA_OPTS="-Xms4G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -
XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -
XX:+PrintGCApplicationConcurrentTime-Xloggc:${GC_LOG_LOCATION}/gc-pinot-broker.log"
${PINOT_DISTRIBUTION_DIR}/bin/start-broker.sh -zkAddress ${ZOOKEEPER_ADDRESS} -
configFileName ${SERVER_CONF_DIR}/broker.con
Perform similar operation for almost all the other services as well
r
Thanks alot @User