hi team, we are using Pinot table with ingestion aggregation enabled. I am wondering how does Pinot handle out of order events in such case? ex: im aggregating data in Pinot for given day. If i receive a record in Pinot today (july 10th) for a event that happened on July 8th, will ingestion aggregation ignore the event or will it continue to process?

Ingestion aggregation is only for rows that have exact same value for dimension and time columns. In your particular case, the row will be ingested, it won’t be pre-aggregaed though. It is not necessarily a bad thing, the answer will still be correct.

but we observed that data to be missing from pinot, is that possible?

do you have an offline table? if it’s later than the time boundary of the hybrid table, it will effectively not be queried. but otherwise it’s what Mayank said. the data will still be there, just not pre-aggregated with the on time events

ah! yeah we do have hybrid table and your reasoning does makes sense. thanks Johan. i want to clarify on this statement though,

> if it’s later than the time boundary of the hybrid table, it will effectively not be queried

do you mean data would exist but it will be ignored during query time because the table is configured as hybrid table? we have a way to directly query *_realtime or *_offline tables, if i explicitly specify query to read from *__realtime table should i expect out-of-order events data to be shown?

exactly what you said. the data should be there, but the broker issues

select from _OFFLINE where time <= X

and

select from _REALTIME where time > X

. if you manually query

your_table_REALTIME

, the data should still be there

that makes sense, thanks again for quick help here Johan. appreciate it!