@kapa.ai mysql CDC connector for Airbyte self-hosted is failing with error

Debezium Configuration Error: The sync encountered a schema mismatch while processing a row.  Please ensure that 'binlog_format' and 'binlog_row_image' are configured correctly, then restart the sync.  For detailed configuration instructions, refer to our documentation: <https://docs.airbyte.com/integrations/sources/mysql>.

<https://dev.mysql.com/doc/refman/8.0/en/binary-log-formats.html>

after couple of successful runs . No change to table schema was made

The error you're encountering with the Airbyte MySQL CDC connector—regarding a schema mismatch and the recommendation to check

binlog_format

and `binlog_row_image`—is a known issue that has been reported by multiple users, even when no schema changes have been made to the tables. Key points from the knowledge sources: - This error often appears after a few successful syncs, and is typically related to how the MySQL binlog is configured or how Debezium (the underlying CDC engine) is able to reconstruct table schemas from the binlog and its internal schema history. - Users have reported that the error can occur even when the recommended settings are in place (

binlog_format=ROW

and

binlog_row_image=FULL

) and no schema changes have been made. The error message may look like:

Error processing before of row in <table> because it's different column size with internal schema size X, but before size 1, restart connector with schema recovery mode.

- The root cause is often related to the loss or corruption of the Debezium schema history, which is required to correctly interpret binlog events. If the schema history is missing or incomplete, Debezium cannot match binlog events to the correct table schema, resulting in the observed error. This can happen after connector restarts, crashes, or if the schema history is not persisted correctly. In Airbyte, this is managed internally and not exposed as a Kafka topic, but the underlying problem is the same as described in Debezium documentation and issues. - Workarounds reported by users include: - Resetting the connection in Airbyte, which forces a new snapshot and rebuilds the schema history. This often resolves the issue temporarily, but it may recur. - Ensuring MySQL settings: Double-check that

binlog_format=ROW

and

binlog_row_image=FULL

are set and remain unchanged. - Checking MySQL server variables: Some users found that the

log_bin_use_v1_row_events

variable, if set to

ON

, can cause issues with CDC. Setting it to

OFF

resolved the problem for some environments, as it changes how MySQL writes the binlog and can affect compatibility with Debezium/Airbyte CDC ([see user report](https://github.com/airbytehq/airbyte/issues/51607)). - If you are using MariaDB, or if your MySQL server is a managed service (like AWS RDS), additional compatibility issues may arise, and the same error pattern has been observed. Summary of recommended actions: 1. Verify MySQL settings: -

binlog_format=ROW

-

binlog_row_image=FULL

- (Optional)

log_bin_use_v1_row_events=OFF

if available and applicable 2. Reset the Airbyte connection to force a new snapshot and schema history rebuild. 3. Monitor for schema changes (even implicit ones, such as triggers or metadata changes) that might not be obvious. 4. If the issue persists: - This may be a limitation or bug in the current Airbyte CDC implementation with MySQL, as several users have reported instability and recurring schema history issues even with correct settings and no schema changes ([see issue discussions](https://github.com/airbytehq/airbyte/issues/51607), [another example](https://github.com/airbytehq/airbyte/issues/41948)). If you continue to experience this error despite following all configuration recommendations, it may be necessary to open a support ticket with Airbyte or follow ongoing GitHub issues for updates and potential fixes. References: - [MySQL CDC Issue - Debezium error - Can't incremental sync after the first full load](https://github.com/airbytehq/airbyte/issues/51607) - [MySQL CDC throws DebeziumException](https://github.com/airbytehq/airbyte/issues/41948) For the most up-to-date configuration instructions, always refer to the official Airbyte documentation: [Airbyte MySQL Source Docs](https://docs.airbyte.com/integrations/sources/mysql).

@kapa.ai 1. mysql settings - ok 2. reset sometimes helps, but its not an option to reload all the tables with 50/50 probability when something went wrong 3. how?

A message from kapa.ai