I’m trying to set up a source from AWS MySQL aurora but getting error: Failure reason: Saved offset no longer present on the server. Please reset the connection, and then increase binlog retention and/or increase sync frequency. See https://docs.airbyte.com/integrations/sources/mysql/mysql-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs for more details.

The error message you're encountering indicates that the saved offset for the incremental sync is no longer present on the server. This typically happens when the binlogs needed for the incremental sync have been removed by MySQL. Here are the steps you can take to resolve this issue: 1. Increase Sync Frequency: - Sync the data source more frequently to ensure that the binlogs are read before they are purged. 2. Increase Binlog Retention: - Set a higher value for

binlog_expire_logs_seconds

. It is recommended to set this value to a time period of 7 days. This will ensure that the binlogs are retained for a longer period, giving Airbyte enough time to read them. - Note that increasing the binlog retention period will require more disk space. Here is the relevant documentation link for more details: [MySQL Troubleshooting - Under CDC incremental mode, there are still full refresh syncs](https://docs.airbyte.com/integrations/sources/mysql/mysql-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs). ### Example Configuration for `binlog_expire_logs_seconds`: You can set the

binlog_expire_logs_seconds

parameter in your MySQL configuration file (

my.cnf

or

my.ini

), or dynamically using the MySQL command line:

sql SET GLOBAL binlog_expire_logs_seconds = 604800;  -- 7 days in seconds

After making these changes, reset the connection in Airbyte and try syncing again. If you continue to face issues, please refer to the detailed troubleshooting steps in the [Airbyte MySQL Troubleshooting Guide](https://docs.airbyte.com/integrations/sources/mysql/mysql-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs). By following these steps, you should be able to resolve the issue and ensure that your incremental syncs work correctly.

@kapa.ai what if I’m using aurora mysql instead

If you're using Aurora MySQL with Airbyte, there are a few important considerations and potential issues you might encounter based on the knowledge sources provided: 1. Compatibility: - Airbyte connectors are compatible with MySQL databases, but using them with Aurora MySQL might not be 100% compatible. You may need to debug further issues on your own ([source](https://discuss.airbyte.io/t/1022)). 2. CDC Replication Issues: - Users have reported issues with Change Data Capture (CDC) replication when using Aurora MySQL. For example, incremental syncs may not pick up new records even though the binlog position is updating ([source](https://discuss.airbyte.io/t/3329)). - Another issue reported is that Airbyte might not identify the cursor field for incremental syncs, leading to errors ([source](https://discuss.airbyte.io/t/1022)). 3. Error Handling: - Specific errors such as

Can't find file: 'host' (errno: 2 - No such file or directory)

have been encountered, which might be related to the underlying database setup or configuration ([source](https://discuss.airbyte.io/t/1022)). 4. Ongoing Improvements: - Airbyte is aware of the challenges in replicating MySQL workloads on Amazon Aurora and is committed to enhancing the reliability of these operations to match those hosted on other platforms ([source](https://github.com/airbytehq/airbyte/issues/28681)). ### Actionable Steps: 1. Debugging: - If you encounter issues, you may need to debug further. Check the logs for specific error messages and investigate accordingly. For example, ensure that the binlog position is correctly updating and that the cursor field is properly identified. 2. Configuration: - Verify your MySQL and Airbyte configurations. Ensure that all necessary settings for CDC are correctly applied. Refer to the [Airbyte MySQL documentation](https://docs.airbyte.com/integrations/sources/mysql#change-data-capture-cdc) for detailed setup instructions. 3. Community and Support: - Engage with the Airbyte community for support. You can refer to existing discussions and issues on the [Airbyte forum](https://discuss.airbyte.io/t/3329) and [GitHub issues](https://github.com/airbytehq/airbyte/issues/20179). 4. Stay Updated: - Keep an eye on updates from Airbyte regarding improvements to the MySQL connector, especially those related to Amazon Aurora. If you need further assistance, you can also check the [Airbyte documentation](https://docs.airbyte.com/) for more detailed guidance. If you have specific logs or error messages, sharing them with the community or support team can help in diagnosing the issue more effectively.