Has anyone gotten

Aurora MySQL (5.7.0) 2.10.2

to work with CDC? Incremental syncs are not picking up new records

@Subodh Chaturvedi (Airbyte) can you help here?

Aurora should work with our implementation. I took a look at the logs attached and I see that no records were returned from MySQL in 5minutes. @gunu can you try increasing the first wait record time out to 20 minutes and see what happens. @Rodi (Airbyte) I think this calls for heartbeat support in mysql cdc as well

thanks for the response @Subodh (Airbyte). trying that out now so will let you know in 20min however i dont think that'll change anything. i had the incremental + dedupe running for a few days and it didn't pick up anything. see screenshot of first sync following by 3-hourly syncs

can you share the logs

d39daee2_294b_4e75_8522_740f14df7112_logs_62915_txt.txt

So we did not get any records from binlogs in 20 minutes as well. This is weird. @gunu are you confident that the binlogs are setup properly. I assume since the check command is passing it is but just in case we are missing something specific about Aurora. Also are the tables

<http://mydb.my|mydb.my>_table_a,<http://mydb.my|mydb.my>_table_b

have change events happening to em?

• i'm not confident the binlogs are setup properly. i think they as i've done it before but it'd be great if we could double check together (or provide me specifics - the existing mysql docs variable names don't seem to match with the cluster param group variables im working with) • the tables definitely have new events coming through • did it work for

Aurora MySQL (5.7.0) 2.10.2

Could you try doing this https://aws.amazon.com/premiumsupport/knowledge-center/aurora-mysql-increase-binlog-retention/#:~:text=For%20other%20Aurora%20MySQL%20compatible,is%202160%20(90%20days).

alright well

binlog retention hours

is set to NULL in the mysql DB 🤦 i configured all the variables via the AWS UI (cluster param groups for the aurora mysql) that's most likely it hey?

not sure. hehe! lets increase it to 7 days and see what happens.

yeah will give that a whirl now and report back these docs might need to be updated i think... i was trying to find

binlog_expire_log_seconds

but couldn't..

yeah RDS and Aurora have different terminologies sometimes.

okay that didn't work. i changed that parameter

binlog retention hours	72	binlog retention hours specifies the duration in hours before binary logs are automatically deleted.

• i changed the binlog retention hours to 72 • i reset the connector with full overwrite to make sure binlog position would be reset • i waited for new records to come in • i synced again --> it didnt pick up the new records

hmmmm! Alright I will give it a try on my end. Can you create an issue please. Am 70% sure we just need to figure out the right params here