Is there any reason why a realtime table would create a segment that immediately is shown as bad?

Hmm, what does external view say? If it also shows as OFFLINE, then the server log should have the error. If it says ONLINE, then may be a UI bug?

Thank you for the reply. So, it's really really hard to figure out when things go wrong. I had inadvertently set realtime.segment.flush.threshold.rows to a really high value and that seems to have caused

Caused by: java.lang.OutOfMemoryError: Direct buffer memory

Unfortunately, because of all the swagger exceptions noise in the journal for that service I didn't see that error. Restarting the service didn't help. Deleting the problematic segment didn't help. What troubles me is that pinot shows that server to be up, the logs for it seemed to be ok, but nothing worked 😞 I even went as far as to delete everything (zookeeper config, data, everything) and try again.

This is really good feedback. What version are you using? We have added some debug endpoints that will make it easier to find these issues, they are part of the upcoming release. We will continue to improve based on feedbacks like these

That's awesome to hear! I've been using 0.7.1 for this poc - I was being lazy and didn't want to upload the master branch build I had made to all the servers. Obviously a bad choice.

If this is a local test and you have the time, I’d be curious to ensure that master’s debug endpoint could have caught the error

I can do that after I finish the kafka stream ingestion ... likely Tues/Weds

Thanks