Anyone faced following error ? Once this happens I have to restart Pinot server and the data during these errors are not consumed by realtime table.

16:20:24.624 SegmentColumnarIndexCreator - Caught exception org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed while refreshing realtime lucene reader for segment: mydata__1__25531__20220323T2114Z

I'm assuming you have text index enabled? cc: @User have you seen this before?

Yes it is text index

Can you post the rest of the stacktrace to get more info here?

It happened again, So I checked logs for all the servers. Found that one of the servers running out of space. I increased the space and the partitions. I am having 9 servers with 4TB disk and 4 partitions ingesting 1TB data per day. I increased the partition from 4 to 8. Do we need more partitions so that all the consuming data is not going to a few servers and causing space issues?

Hi Matt -- I can take a look sometime today

Thanks @User