Hi team, Can I use upsert in realtime table and use realtimetoofflinesegmentstask at the same time? will this task make sure rows in offline table upserted?

To my best knowledge, I don’t think the task does upsert when moving segments to offline table. Under the hood, it downloads the raw segments of realtime table from the deep store, applies transformations and indices defined in the offline table and then uploads the segments.

I noticed dedup is based on all column values are same. Does it support primary key?😅

Yeah it’s full row right now. Pk based dedup is not supported here yet.

Got it. Thanks.