Is Logical Replication for Postgres real-time? Or ...
# ask-community-for-troubleshootingj
Justin Sharp
10/19/2021, 3:12 AMIs Logical Replication for Postgres real-time? Or is it near-real-time due to some set interval? If the latter, how small can that interval be?
For context: we need CDC on our monolith Postgres DB, so that we can stream table changes to Kafka topics to power downstream microservices. Debezium is being investigated. Would be wicked if Airbyte Cloud could be used instead.
✅ 1
h
Harshith (Airbyte)
10/19/2021, 4:21 AM
Hey Justin you can check the usecases here https://docs.airbyte.io/integrations/sources/mysql#change-data-capture-cdc
Harshith (Airbyte)
10/19/2021, 4:24 AM
j
Justin Sharp
10/19/2021, 4:30 AM@Harshith (Airbyte) This is for MySQL, and like the PostgreSQL documentation, nothing is specific on how "real-time" the CDC implementation is.
Justin Sharp
10/19/2021, 4:32 AM@Harshith (Airbyte) What I would like to understand is the smallest value the "sync interval" can be.
h
Harshith (Airbyte)
10/19/2021, 8:39 AM
We have a sync frequency as lowest as 5 minutes in incremental/full refresh
j
Justin Sharp
10/19/2021, 2:49 PMThanks @Harshith (Airbyte)! Airbyte won't be feasible then for our real-time needs 😞
Justin Sharp
10/19/2021, 3:19 PM@Harshith (Airbyte) Do you know of any desire to support our use-case? Supporting event-driven microservices through a more "real-time" replication of PostgreSQL is a popular use-case given the traction of tools like Debezium.
h
Harshith (Airbyte)
10/19/2021, 4:05 PM
@[DEPRECATED] Marcos Marx can you help here
u
[DEPRECATED] Marcos Marx
10/19/2021, 4:24 PM
Right now Airbyte doesn't support real-time integration or webhooks. There is an issue requesting this feature (Postgres CDC real-time with Kafka). Airbyte is a tool where you can plug any source to any destination and to achieve this in early stage the plan was develop batch inserts. Maybe is a future feature implementation now that Airbyte has a good amount of source/destinations.
👍 1
3 Views