We had to rush the upgrade to 0.9.1 because of the log4shell issue and ran into this change afterwards: https://github.com/apache/pinot/pull/7523 We are using the realtimeToOffline task with an upsert based table and using a rollup based on our business rules to get the same results in the offline segments. This change makes it impossible to keep the same config if we want to change the table config.

Side note, we released 0.9.2 to fix the second vulnerability with log4j2

Our usecase is updating some metric fields with upserts in a limited window (dimensions are unchanged) and these changes are always bigger than the previous value so we use a max rollup on these metrics to achieve the same results.

i guess this is similar to what this PR is trying to address: https://github.com/apache/pinot/pull/7733 ?

yes

let me quickly add a skipping mechanism for this. be aware that enabling this would mean it is user's responsibility to ensure changing data are not to be touched by the r-to-o job.

cc: @Jackie