A question I've never seen really answered is, can we use Airbyte to CDC from PostgreSQL (log based CDC) into Snowflake, in real time? (ie, <= 30 seconds from data being generated in postreSQL to it being available in Snowflake)
q
Qamarudeen Muhammad
12/12/2022, 3:17 PM
I am not sure, you can get less than 120s as in mentioned in Airbyte documentation. There are dockers that need to boot up at every sync
👀 1
j
Jay Jay
12/12/2022, 3:52 PM
Ouch, since it was using Debeezium behind scenes I thought maybe it would stay up and running in real time like Debeezium does.
u
user
01/02/2023, 8:45 PM
Hello Jay Jay, it's been a while without an update from us. Is the real time CDC feature one that you critically need?
j
Juan Sisti
01/03/2023, 7:01 PM
Hey @Saj Dider (Airbyte) I would say so.
I do need something that can get the data synced in less than 60 seconds, right now we are evaluating spinning up more airbyte instances to make each sync a subset of tables so we can get the data faster tho.
s
Saj Dider (Airbyte)
01/10/2023, 5:40 PM
Thanks @Juan Sisti, I sent the feedback here to the team