Hi there! Wanted to confirm behavior of Airbyte in the below scenario:
Airbyte Version: 0.39.41-alpha
Source: PostgreSQL, connector version: 1.0.21
Destination: Snowflake, connector version: 0.4.38
Sync Mode: Tried both (incremental/dedup+History and Incremental/append)
Replication Method: Tried both (cursor based and cdc based)
Transformation: Tried both (Raw Data and Normalized)
Behavior: If my sync frequency is every 30 minutes, I see that Snowflake warehouse is started, bunch of queries are executed EVEN IF THERE ARE NO UPDATES AT PostgreSQL end. It shows 0 Bytes | no records | no records | 5m 35s | Sync.
Question: Why does the destination Snowflake instances needs to be touched if there are no updates at the source end. It doesn't make any sense.
Drawback: For our use-case where source is updated very infrequently. We end-up paying for 5 minutes of warehouse every 30 minutes even though we hardly sync 100-200 records.
Am I doing something wrong here? Is the understanding correct? Thanks!