@kapa.ai what happens if there's an ongoing sync in the background that I didn't know of, then I migrated the database to from RDS instanceA to instance B.. what will happen? will there be data lost?

Hmm, I don't know enough to give you a confident answer yet. The behavior in such a scenario would depend on various factors such as the sync mode you're using, the state of the sync when the migration occurred, and how the specific connectors handle interruptions and changes in the source or destination. However, here's what I can tell you based on the knowledge sources: 1. If a sync fails, you won't lose data, but no data will be added or updated in your destination. Airbyte will automatically attempt to replicate data 3 times. If a sync fails, you can see and export the logs for those attempts in the connection detail page. Once the issue causing the sync failure is resolved, Airbyte resumes your sync from where it left off. [source](https://discuss.airbyte.io/t/77) 2. If a connector crashes in the middle of a sync, it depends on the connector and the cause of the crash. If the connector is able to checkpoint its state, then you may have what is called a partial sync failure (some records are synced but not all). Most likely though, you'd want to retry the sync anyway in the case of a failure. [source](https://github.com/airbytehq/airbyte/discussions/33059) 3. For full refresh syncs, the mechanism by which a destination connector accomplishes the full refresh will vary from destination to destination. For certified database and data warehouse destinations, Airbyte recreates the final table each sync. This allows leaving the previous sync's data viewable by writing to a "final-table-tmp" location as the sync is running, and at the end dropping the old "final" table, and renaming the new one into place. [source](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/full-refresh-overwrite-deduped#destination-specific-mechanism-for-full-refresh) 4. In the case of incremental syncs, if the source schema changes (for example, when a column is added, renamed or deleted from an existing table etc.), it is recommended to trigger a Full refresh - Overwrite to correctly replicate the data to the destination with the new schema changes. [source](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/incremental-append#known-limitations) For a more accurate answer, I would recommend checking the Airbyte [documentation](https://docs.airbyte.com/) or reaching out to the Airbyte community on their [discussion forum](https://discuss.airbyte.io/) or [Slack](https://slack.airbyte.io/).

can I pause a sync job in airbyte OSS? @kapa.ai

A message from kapa.ai

Based on the information available in the knowledge sources, there is currently no explicit feature in Airbyte Open Source (OSS) that allows you to pause a sync job in the middle of its operation. This has been discussed in the community [here](https://github.com/airbytehq/airbyte/discussions/32739) and [here](https://github.com/airbytehq/airbyte/issues/32587), but as of the latest updates, no such feature has been implemented. However, you can disable a connection which will stop the syncs from running. But once you disable the connection, the cancel sync button disappears. This issue was resolved and the sync will stop once you disable your connection as mentioned [here](https://github.com/airbytehq/airbyte/issues/16905). For more control over when syncs are triggered, you could potentially use the Airbyte API to manually trigger syncs when desired, as suggested in this [forum post](https://discuss.airbyte.io/t/379). Please note that these are workarounds and may not provide the exact functionality you're looking for. It's always a good idea to check the latest Airbyte [documentation](https://docs.airbyte.com/) or reach out to the Airbyte community for the most up-to-date information.

@kapa.ai regarding this: "If a sync fails, you won't lose data, but no data will be added or updated in your destination. Airbyte will automatically attempt to replicate data 3 times. If a sync fails, you can see and export the logs for those attempts in the connection detail page. Once the issue causing the sync failure is resolved, Airbyte resumes your sync from where it left off." What happens if it failed 3 times? Will airbyte retry at later time?

A message from kapa.ai

Yes, Airbyte has a retry mechanism in place for sync jobs. If a sync job fails, Airbyte will retry the pipeline. Each of these sub-invocations of a job is called an attempt. By default, Airbyte is configured to allow the following: - 5 subsequent attempts where no data was synchronized - 10 total attempts where no data was synchronized - 20 total attempts where some data was synchronized After an attempt where no data was synchronized, Airbyte implements a short backoff period before starting a new attempt. This backoff period will increase with each successive complete failure. For example, it will backoff for 10 seconds after the first complete failure, 30 seconds after the second, 90 seconds after the third, and 4 minutes and 30 seconds after the fourth. If a sync job fails after all these attempts, it will wait for the next scheduled sync to retry. You can read more about Airbyte's retry approach [here](https://docs.airbyte.com/understanding-airbyte/jobs#retry-rules). Please note that these values are configurable for open-source users. You can configure them as per your requirements. See [Configuring Airbyte](https://docs.airbyte.com/operator-guides/configuring-airbyte#jobs) for more details.