https://pinot.apache.org/ logo
m

Mohamed Sultan

04/21/2021, 3:12 PM
Hi team, I'm just facing this issue when i do run query.
d

Daniel Lavoie

04/21/2021, 3:13 PM
Your pinot broker is probably not healthy. Can you check the running status of your pods?
m

Mohamed Sultan

04/21/2021, 3:14 PM
I just tried deleting the broker and restarted
and pinot stopped consuming data from kafka as well
for all tables
d

Daniel Lavoie

04/21/2021, 3:16 PM
If your cluster is not healthy nothing will work as expected.
m

Mohamed Sultan

04/21/2021, 3:17 PM
any other way to sort out?
d

Daniel Lavoie

04/21/2021, 3:17 PM
What is the running status of your pods?
m

Mohamed Sultan

04/21/2021, 3:18 PM
it's in running state
d

Daniel Lavoie

04/21/2021, 3:18 PM
Any error in the logs of each pods?
m

Mohamed Sultan

04/21/2021, 3:18 PM
This is for broker-0
d

Daniel Lavoie

04/21/2021, 3:19 PM
what about the others?
m

Mohamed Sultan

04/21/2021, 3:21 PM
This is controller-0
server-0
kindly do favor.
d

Daniel Lavoie

04/21/2021, 3:25 PM
Seems like you have a schema error regarding a dateformat. Can you access the Cluster Manager UI and check if broker is live? Finally, which k8s provider are you running on?
k

Kishore G

04/21/2021, 3:25 PM
Looking at the exception in the log that the time stamp format in the Kafka message does not match what you have in table schema
d

Daniel Lavoie

04/21/2021, 3:25 PM
Schema error is a different problem.
m

Mohamed Sultan

04/21/2021, 3:25 PM
GKE
d

Daniel Lavoie

04/21/2021, 3:26 PM
I would suggest fixing your schema, then trying the query again. The error you got might be caused by a broker being restarted
m

Mohamed Sultan

04/21/2021, 3:26 PM
brokers are in alive status
ok thats fine. but some table were consuming data. But now it got stopped.
can you point me which schema has error?
m

Mayank

04/21/2021, 4:30 PM
The lastest image you pasted has an error that indicates a bad date Feb 29 2021 (this year only had 28 days in Feb)
e

Elon

04/21/2021, 9:37 PM
If you tail the server logs do you see
java.lang.OutOfMemoryError: Direct buffer memory
? The servers may be running but the direct memory may be hitting the limit.