Hi guys, I’m new to pinot. I have a realtime table...
# troubleshootinga
Andrew Sunarto
08/16/2022, 1:09 AMHi guys, I’m new to pinot. I have a realtime table filled with 2603 documents from a Kafka producer, which continues to produce messages to the Kafka topic. The number of documents in pinot from this topic is no longer increasing, so I’m wondering what possibly could have caused this. Reported/Estimated size for this table is 0 bytes (although there are 2603 valid documents in the table) and the status for each of the table segments is Good and the server is Consuming in the replica set. Any ideas I can try to get pinot to continue collecting the messages from the kafka sink?
n
Neha Pawar
08/16/2022, 1:14 AM
it sounds like the pinot table consumed from all the partitions, and hit some error when it tried to build a segment using those consumed rows to flush it to disk. The reported size will only calculate using ONLINE segments (it skips CONSUMING), so your ideal state probably has all segments in CONSUMING state.
Neha Pawar
08/16/2022, 1:14 AM
can you chek the pinot-server logs for any exceptions related to segment build?
Neha Pawar
08/16/2022, 1:14 AM
also, what’s the ideal state and external view looking like?
Neha Pawar
08/16/2022, 1:15 AM
most typically, segments trying to go from CONSUMING to ONLINE state encounter exceptions because of some time column issues.
a
Andrew Sunarto
08/16/2022, 1:19 AMah okay, I do see this in the server logs
Caught exception while transforming data type for column: timestampAndrew Sunarto
08/16/2022, 1:22 AMFor some reason, the datetime was changed to only include date and time was removed altogether. Adding time back to the datetime fixed the messages not showing up in pinot. Appreciate the help!
👍 1