Hi pinot community. I have a general question about realtime upsert table and merge-rollup. Documentation says that MergeRollupTask is supported on REALTIME tables, without upsert or dedup. https://docs.pinot.apache.org/operators/operating-pinot/minion-merge-rollup-task If one has an upsert table, with upsert/metadataTTL set to 3d. That means that upserts are limited to 3d buffer of young data, correct ? And then, let’s consider that merge rollup task is desired for data older than 1 week. So, there is no interference possible between upsert and merge-rollup. However, it is not possible to have a realtime table schema/config. What is wrong with this use case ?

I think this is a valid use case, and if I am not wrong this is either being worked on or in the roadmap @Manish?

Yes this is a valid usecase, where we want mergeRollup on historical/older data (offline table) where we can also use things startree index, whereas realtime is upsert data/there's clear separation. Not being worked on yet but this is in the roadmap.

I have noticed a pinot recipe here: https://dev.startree.ai/docs/pinot/recipes/upserts-real-time-offline-job that indicates that upsert and RealTimeToOfflineTask is possible. Then I guess: MergeRollupTask can be configured on the Offline table. Correct ?

Yes MergeRollupTask can be configured on offline table. It cannot be configured on realtime table which is upsert enabled. https://docs.pinot.apache.org/operators/operating-pinot/minion-merge-rollup-task (from doc)

REALTIME tables, without upsert or dedup

`