I have a realtime table with

"realtime.segment.flush.threshold.rows": "10000000",
        "realtime.segment.flush.threshold.time": "6h",
        "realtime.segment.flush.threshold.segment.size": "400M",

I changed these values two days ago, previously the "rows" limit was 0. Pinot is generating segments with 3,333,333 rows, every ~90 minutes, 95-100MB -- significantly below any of the limits. Server logs show

Starting consumption on realtime consuming segment ... maxRowCount 33333

and

Stopping consumption due to row limit nRows=3333333

- I am trying to figure out where that limit is coming from.

10000000

/ numPartitionsOnServer

So

realtime.segment.flush.threshold.rows

is the a threshold total for all the partitions on the server - I would not have guessed that - thank you

Also, I see you've set size threshold 400M. For that to take effect, you need to set rows threshold to 0

I assumed these were OR limits - when one limit was reached, the segment was completed. Thanks again - I am learning more.

cc: @User For any docs improvements

Thanks. it looks like that doc does not mention the

#rows / numPartitionsOnServer

point

ah yeh good point, will update