https://linen.dev logo
p

Peter

02/14/2022, 5:13 PM
Hi, Just a question which I haven't been able to answer by myself, for the Postgres source connector, in the Incremental + Deduped History mode, the deduplication is performed at the normalization step, right? So if I choose not to have the basic normalization step, I would have to implement the deduplication logic on my side. If so, is there a recommended way to do this? For example using ROW_NUMBER or DISTINCT ON ?
c

Cayden Brasher

02/16/2022, 2:46 AM
Yes, you need to implement the dedup process yourself. The best way probably is grouping by the primary key from your source.
r

Roxy Trofymchuk

02/16/2022, 10:23 AM
Ok thanks! 🙂
2 Views