Hello, everyone. My team and I are looking at Airb...
# ask-community-for-troubleshooting
s
Hello, everyone. My team and I are looking at Airbyte for some data integration work flows we are building. One of our potential needs is low latency syncing between two data stores (SLA in minutes). I understood that Airtbyte Cloud can sync source to destination as often as 1 hour. Can the self-hosted application perform more frequent incremental syncs? I've tried using the cron scheduler with a cron expression to run every 10 minutes but from the UI it doesn't seem that the schedule is being used. Additionally, we were also curious to know if the source is a stream like Kafka does a sync period still apply or can Airbyte manage continuous syncing from Kafka to a database destination?
k
A message from kapa.ai
Yes, with the self-hosted Airbyte application, you can perform more frequent incremental syncs. Although the sync frequency in the UI can be set as low as five minutes, you can trigger Airbyte syncs more frequently using the API or integrating it with a workflow management tool like [Airflow](https://airbyte.com/blog/airflow-etl-pipelines), [Prefect](https://airbyte.com/recipes/elt-pipeline-prefect-airbyte-dbt), or [Dagster](https://airbyte.com/recipes/orchestrate-data-ingestion-and-transformation-pipelines) [source](https://airbyte.com/tutorials/real-time-data-analytics-pipeline). Regarding syncing from a Kafka source, Airbyte can sync data from Kafka to a database destination. Each stream is written to the corresponding Kafka topic [source](https://docs.airbyte.com/integrations/destinations/kafka). However, the sync frequency still applies, and you would need to configure the sync mode and frequency according to your requirements. You can also trigger manual syncs from the Airbyte UI or use the API for more control over the sync process [source](https://airbyte.com/tutorials/mysql-cdc-to-kafka). Keep in mind that Airbyte's incremental synchronization is designed to efficiently keep a source and a destination in sync by replicating only the changes that have occurred since the previous sync [source](https://airbyte.com/tutorials/incremental-data-synchronization).