https://pinot.apache.org/ logo
Join Slack
Powered by
Hi I'm creating table using Pinot UI, I followed t...
# troubleshooting
j
Hi I'm creating table using Pinot UI, I followed the example https://docs.pinot.apache.org/basics/getting-started/pushing-your-streaming-data-to-pinot and copied the table into UI, but when I save the table, it says "Invalid table config: transcript_REALTIME" can anyone help me?
n
were all the previous steps successful - cluster setup, kafka setup and schema creation?
do you have access to the pinotController.log? There should be exception message in there
j
Yes, I was deploying pinot on K8s and creating table using restAPI provided by Pinot. the error is
Copy code
{
  "code": 500,
  "error": "org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata"
}
n
that means kafka is not reachable
you need to change the 
"stream.kafka.broker.list": "localhost:9876",
according to your kafka cluster
j
which file should I change? I just followed the steps in https://docs.pinot.apache.org/v/release-0.4.0/basics/getting-started/kubernetes-quickstart#4-query-using-pinot-data-explorer and creating kafka clusters using
Copy code
helm install -n pinot-quickstart kafka incubator/kafka --set replicas=1
n
@Xiang Fu ^^
what is the output of 
kubectl get all
?
j
NAME           READY  STATUS  RESTARTS  AGE
pod/kafka-0       1/1   Running  0     46m
pod/kafka-zookeeper-0  1/1   Running  0     46m
pod/kafka-zookeeper-1  1/1   Running  0     46m
pod/kafka-zookeeper-2  1/1   Running  0     45m
pod/pinot-broker-0    1/1   Running  1     50m
pod/pinot-controller-0  1/1   Running  0     50m
pod/pinot-server-0    1/1   Running  1     50m
pod/pinot-server-1    1/1   Running  1     50m
pod/pinot-zookeeper-0  1/1   Running  0     50m
NAME                TYPE      CLUSTER-IP    EXTERNAL-IP  PORT(S)           AGE
service/kafka            ClusterIP   10.102.3.210   <none>    9092/TCP           46m
service/kafka-headless       ClusterIP   None       <none>    9092/TCP           46m
service/kafka-zookeeper       ClusterIP   10.108.54.111  <none>    2181/TCP           46m
service/kafka-zookeeper-headless  ClusterIP   None       <none>    2181/TCP,3888/TCP,2888/TCP  46m
service/pinot-broker        ClusterIP   10.107.199.233  <none>    8099/TCP           50m
service/pinot-broker-external    LoadBalancer  10.96.55.75   <pending>   8099:30241/TCP        50m
service/pinot-broker-headless    ClusterIP   None       <none>    8099/TCP           50m
service/pinot-controller      ClusterIP   10.105.81.75   <none>    9000/TCP           50m
service/pinot-controller-external  LoadBalancer  10.107.192.144  <pending>   9000:32385/TCP        50m
service/pinot-controller-headless  ClusterIP   None       <none>    9000/TCP           50m
service/pinot-server        ClusterIP   10.109.0.248   <none>    8098/TCP           50m
service/pinot-server-headless    ClusterIP   None       <none>    8098/TCP           50m
service/pinot-zookeeper       ClusterIP   10.108.40.150  <none>    2181/TCP           50m
service/pinot-zookeeper-headless  ClusterIP   None       <none>    2181/TCP,3888/TCP,2888/TCP  50m
NAME                READY  AGE
statefulset.apps/kafka       1/1   46m
statefulset.apps/kafka-zookeeper  3/3   46m
statefulset.apps/pinot-broker    1/1   50m
statefulset.apps/pinot-controller  1/1   50m
statefulset.apps/pinot-server    2/2   50m
statefulset.apps/pinot-zookeeper  1/1   50m
seems like all clusters running well
n
in your table config json, change the kafka broker address from “localhost:9876” to “kafka:9092"
j
Yes! that works! thank you.
One more question, after creating schema and table, when I publish data into kafka, `kubectl -n pinot-quickstart exec kafka-0 -- kafka-console-producer --broker-list kafka:9092 --topic transcript-topic < /tmp/pinot-quick-start/rawData/transcript.json`I cannot query them in the UI, seems like there was no data. Do you have some ideas?
n
can you query the kafka topic using kafka-console-consumer to confirm if the topic has data? https://kafka-tutorials.confluent.io/kafka-console-consumer-producer-basics/kafka.html
j
Seems like no. Sorry, I'm new to Pinot and Kafka. I used 
kubectl -n pinot-quickstart exec kafka-0 -- kafka-console-consumer --bootstrap-server kafka:9092 --topic transcript-topic --from-beginning
and the responds shows 
WARN [Consumer clientId=consumer-1, groupId=console-consumer-65502] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
x
is this file 
/tmp/pinot-quick-start/rawData/transcript.json
in your local or on that container
you may need to copy that file into container then run that command
j
Thanks for replying! It is in my local. I run 
kubectl -n pinot-quickstart cp /tmp/pinot-quick-start/rawData/transcript.json kafka-0:/tmp/transcript.json
to copy that file and then 
kubectl -n pinot-quickstart exec kafka-0 -- kafka-console-producer --broker-list kafka:9092 --topic transcript-topic < /tmp/transcript.json
but got -bash: /tmp/transcript.json: No such file or directory
Solved. Thanks!
x
cool
you can actually run 
kubectl -n pinot-quickstart exec -it kafka-0 -- bash
to enter into the container to do job
j
@Xiang Fu Thanks! May I ask where can I configure the version of Pinot in helm file? I found helm deploys 0.7 version of Pinot. Though that is not officially released.
x
check the docker image tag
Open in Slack
PreviousNext
Powered by