Hello Everyone, Can we use log based replication in near real time(mysql to mysql) in Airbyte? As of now i can see the minimum batch time required is 5 minutes. I want to do something similar to Streamsets replication in real time

loading...

Sync's have two types, manual or schedule. Currently the smallest schedule is 5 minutes, and its based on the last sync, so if you schedule a new sync at 5 min, it runs, 2 mins pass and you manually run it, then 3 mins pass, this next sync is skipped until 5 mins pass so it would run at the 7th min. If you need something more real time than 5 mins, you should be looking at sensors and orchestrating custom ingestion through something like airflow or dagster.