Hi team, my hybrid table is designed to push 5 days old data from realtime to offline. But I need to backfill 2 days old data. Can I delete the realtime segment and backfill 2 days old data directly as an offline segment? Will that create any side-effects or minion exceptions?

Yes.. it should work.. you don’t have to delete real-time data

Hi @Kishore G, yes there is a minion task that runs everyday to push realtime to offline. The 2 days old data has some wrongness to it. So should I wait for minion to push it to offline and then drop that segment, or can I drop the segment in realtime and backfill in offline directly? I guess one of the concerns is will the minion continue working un-interrupted the next day even if it finds 1 day’s data missing in realtime table

Yeah.. it requires some careful orchestration