Hello everyone , this is continuation of previous question, the thread has reached max no. of replies, so asking outside the thread to start a new one. Destination space is doubled or more than that when using incremental deduped+history mode Why it need table_scd ? Using replication key (timestamp), why cant it pick just updated rows and UPsert/INsert them instead of maintaining table_scd ?

A message from kapa.ai

@kapa.ai Incremental append mode does not serves the purpose but. My usecase matches the outcome of incremental deduped+history, but I dont need the history _scd table, I dont need to track the underlying changes, I just need the final updated row

A message from kapa.ai

@kapa.ai Agreed, your replies are nice but I want to reduce the space or size of destination volume. I can safely ignore it but I cannot ignore the fact that it takes storage and that too, minimum double size of source

A message from kapa.ai

If i manually retain only the latest instance of each record in the history table, will it cause any issue in sync or airbyte ? @kapa.ai

A message from kapa.ai