Hello everyone, I am attempting to set up a conne...
# replication-troubleshooting
m
Hello everyone, I am attempting to set up a connection between Shopify and BigQuery, but am encountering issues. When enabling the connection, the sync takes approximately 15 hours and ends in failure with the message: "_Failure Origin: replication, Message: Something went wrong during replication._" In BigQuery, I am not seeing any data except for a few _airbyte_tmp_ tables. Has anyone else encountered this problem? I have included a text file with the logs and have specified that the replication is set to incremental for all tables (deduped and history) except for the shop table, which is set to full refresh.
o
@Mahdi Rabbani The final tables are going to be created after the sync is finished. You've canceled it. I don't see any issues occurred. Please proceed the sync and wait until the data is fetched completely. Please note: • use the latest versions for the
source
and
destinations
• the initial sync will take a long time (that's normal), all subsequent syncs should be fast.
n
Thank you for your input, Oleksandr! Mahdi, sometimes a sync errors out saying that 'this is normal if the job was cancelled'. Just to double check - you did not cancel the job, correct? In that case one of the connections could have timed out. Could you check any
.env
variables in regards to extending timeouts?
🙏 1
m
Hi, No i did not cancel the job. Where do i check for the .env variables? And could the VM-instance in GCP have something to do with it? By looking at the memory and cpu utilisation, they both seem to have a large increase above 100% during the time of the error i Airbyte
n
Here's a doc on configuring Airbyte's environmental variables, including timeouts, please take a look: https://docs.airbyte.com/operator-guides/configuring-airbyte/