Hello! A couple weeks ago we had a very strange o...
# replication-troubleshooting
n
Hello! A couple weeks ago we had a very strange occurrence with our sync from mysql 5.6 to bigquery. After a particular “full refresh - overwrite” connection failed to sync 3 consecutive times, subsequent syncs of the same connection began taking double the time to complete. Looking at the logs, the extra time was due to an unexplained gap during normalization. After all the sql scripts were written, there would be an hour of nothing before any of them would start executing. However, eventually dbt would successful complete. Even more strangely, a later sync of the same connection failed 3 consecutive times - afterward subsequent syncs once again took the original amount of time. The sync in question is relatively large (5.73 GB) and copies >200 tables. I’ve included a copy of the logs for one run during the period that there was an hour gap in normalization. We’re on airbyte version 0.40.17 running on GKE. We’re happy the this problem fixed itself but we would like help identifying what happened here and how we can prevent it from happening again. Thanks!!
u
Hello Noah Selman, it's been a while without an update from us. Are you still having problems or did you find a solution?