vishal
11/28/2022, 6:55 AM"error": "RealtimeToOfflineTask doesn't support upsert table!"
can't we use upsert with the reltime to offline flow?Seunghyun
11/28/2022, 7:11 AMSeunghyun
11/28/2022, 7:12 AMvishal
11/28/2022, 7:12 AMvishal
11/28/2022, 7:14 AMSeunghyun
11/28/2022, 7:17 AMSeunghyun
11/28/2022, 7:18 AMvishal
11/28/2022, 7:21 AMSeunghyun
11/28/2022, 4:11 PMvalidationTypesToSkip=ALLSeunghyun
11/28/2022, 4:12 PMThe main thing to keep in mind when combining these features is that upsert functionality only applies to real-time tables.
As soon as those segments are moved to an offline table, the upsert logic is no longer applied at query time. We will need to backfill the offline segments created by the real-time to offline job to achieve upsert-like queries.
The above is the caveat from the documentation