Hello @kapa.ai, One of our sync jobs suddenly started failing with a not so clear error message. It simply says

Source process read attempt failed

. Taking a look at the logs, it seems like the job is stuck on an infinite loop. The following 3 messages are very frequent, until the job eventually fails:

2024-10-28 19:11:26 source > INFO pool-2-thread-1 i.a.c.i.d.AirbyteDebeziumHandler$CapacityReportingBlockingQueue(reportQueueUtilization):48 CDC events queue stats: size=0, cap=10000, puts=97, polls=0
2024-10-28 19:11:26 source > INFO main i.a.c.i.d.i.DebeziumRecordIterator(computeNext):87 CDC events queue poll(): blocked for PT10.022736437S after its previous call which was also logged.
2024-10-28 19:11:26 source > INFO main i.a.c.i.d.i.DebeziumRecordIterator(computeNext):140 CDC events queue poll(): returned a heartbeat event: no progress since last heartbeat.

I have refreshed the connection multiple times, but it always fails on the next sync. Can someone help me figure out what the problem is? Attached you’ll finde the attempt logs. Airbyte version: v0.63.13 Source: Postgres - v3.6.22 Destination: Postrges - v2.4.0 (edited)

Based on the information provided in your question and the knowledge sources available, it appears you're experiencing an issue with a CDC (Change Data Capture) enabled sync job for a Postgres source connector. The error message "Source process read attempt failed" and the log patterns you've shared are consistent with issues reported in similar scenarios. From the logs you've provided, we can see that: 1. The CDC events queue is not receiving any new events (size=0, polls=0). 2. The DebeziumRecordIterator is repeatedly blocking for about 10 seconds. 3. Only heartbeat events are being returned, indicating no actual data changes are being captured. This behavior suggests that the CDC process is stuck in a loop, unable to capture new changes from the source database. While I don't have enough information to pinpoint the exact cause of this issue in your specific case, here are a few potential reasons and steps you could take: 1. Check if there have been any changes to the source database configuration, especially related to replication slots or publication settings. 2. Verify that the CDC-enabled tables in your source database haven't undergone any schema changes that might be incompatible with the current connector configuration. 3. Ensure that the replication slot used by Airbyte hasn't been dropped or altered in the source database. 4. Try resetting the CDC state for the connection. This can be done by going to the connection page, clicking on "Reset Stream" and selecting the option to reset the CDC state. 5. If possible, check the Postgres server logs for any errors related to replication or the specific replication slot used by Airbyte. 6. Consider upgrading your source-postgres connector to the latest version, as there might be bug fixes related to CDC processing in newer versions. If these steps don't resolve the issue, you may need to provide more detailed logs and information about your specific setup to the Airbyte team for further investigation. You can do this by opening an issue on the [Airbyte GitHub repository](https://github.com/airbytehq/airbyte/issues) with all relevant details and logs. Remember to always backup your data and test any changes in a non-production environment first. [Github Issue #42401](https://github.com/airbytehq/airbyte/issues/42401) shows similar log patterns to what you're experiencing, which suggests this might be a known issue with the CDC implementation for the Postgres source connector.