This message was deleted.

Hi Oliver, Do you have late arrival data? If so, then you are ingesting for multiple time chunks at any given time ... when one time chunk persists, they all have to persist. Also, the thresholds here apply to the sum of the data in the row buffers, not just the largest one. Could the above explanation be what's happening in your case?

Sorry not sure to get your point no. What means the

(estimated) bytes[691,997,144]

? Is it the estimated size of what it is going to be persist on disk ?

are you using sketches by any chance? They tend to over-estimate the memory

No, no tethasketches

What means the

(estimated) bytes[691,997,144]

According to the code it is the row buffer size:

// NB: The rows are still in memory until they're done persisting, but we only count rows in active indexes.
    rowsCurrentlyInMemory.addAndGet(-numPersistedRows);
    bytesCurrentlyInMemory.addAndGet(-bytesPersisted);

    <http://log.info|log.info>("Persisted rows[%,d] and (estimated) bytes[%,d]", numPersistedRows, bytesPersisted);

I'm not sure why it is considered an estimate ... I see (estimated) used in multiple places ...

ah, the estimated memory usage is for in-memory data prior to persist