Hi all, has anyone ever had issues with EC2 instan...
# ask-community-for-troubleshooting
Hi all, has anyone ever had issues with EC2 instances getting into a state of "Instance reachability check failed" when running airbyte? The only way for me to recover is to restart the instance (I can't even SSH in). I'm trying to sync ~50m records from MySQL to BigQuery. If I select only 1 or 2 tables, then it works fine, but as soon as there are more tables, then it falls over. CPU utlization peaks at ~60%. I've tried with a t3.medium and t3.large, and I've tried changing the GCS Staging from 5mb to 50mb. I can see that the
tables have been created in BigQuery, and that's it
βœ… 1
OK, to me it looks like the server is running out of memory. I'm going to try a t3.xlarge and see if that solves it πŸ™
Did the t3.xlarge work?
It’s complicated to estimate the right instance size based that your data can be large jsons or wide-column tables 😞
Yep, xlarge worked @Marcos Marx (Airbyte)!
βœ… 1
I was monitoring memory usage, and it used about 8-9GB memory, so around half of the xlarge
About 55m records in total, so not huge at all, and we don't have particularly wide tables. I'm surprised it used that memory, but I guess that's java for you
I wonder if t3.large will be enough, or if that will fall over as well