I am positive I am missing something here. There a...
# generals
Subbu Subramaniam
11/25/2019, 2:45 AMI am positive I am missing something here. There are no messages produced on the topic (therefore, nothing to consume for the realtime consumers), and yet, when you issue a query, the realtime servers crash. Is that correct? How many offline segments do you have?
a
Alex
11/25/2019, 6:16 PM0 offline segments. Table was built off kafka topic using realtime segments only (batch job dumped data into kafka and then Pinot was pointed to consume from it)
Alex
11/25/2019, 6:17 PMWhen query is issued, I can see that consumers are entering restart loop (or it least it looks like that based on logs)
s
Subbu Subramaniam
11/26/2019, 9:21 PMpinot consumers will re-create kafka consumers every few minutes if there are no incoming messages during that interval. This is to prevent us from getting stuck to some old kafka broker that for some reason may not produce messages. As long as you don't see a WARN or ERROR log, it shoiuld be OK. This in particular should be an INFO log.
a
Alex
11/26/2019, 10:21 PMOh, maybe that is what happening
Alex
11/26/2019, 10:21 PMWill start slowly writing to doublecheck