Hi all, has anyone ever had issues with EC2 instan...
# ask-community-for-troubleshootingj
jonty
02/16/2022, 12:58 PMHi all, has anyone ever had issues with EC2 instances getting into a state of "Instance reachability check failed" when running airbyte? The only way for me to recover is to restart the instance (I can't even SSH in). I'm trying to sync ~50m records from MySQL to BigQuery. If I select only 1 or 2 tables, then it works fine, but as soon as there are more tables, then it falls over. CPU utlization peaks at ~60%. I've tried with a t3.medium and t3.large, and I've tried changing the GCS Staging from 5mb to 50mb. I can see that the
_airbyte_tmp...✅ 1
jonty
02/16/2022, 2:29 PMOK, to me it looks like the server is running out of memory. I'm going to try a t3.xlarge and see if that solves it 🙏
m
Marcos Marx (Airbyte)
02/16/2022, 7:55 PMDid the t3.xlarge work?
Marcos Marx (Airbyte)
02/16/2022, 7:55 PMIt’s complicated to estimate the right instance size based that your data can be large jsons or wide-column tables 😞
j
jonty
02/16/2022, 9:06 PMYep, xlarge worked @Marcos Marx (Airbyte)!
✅ 1
jonty
02/16/2022, 9:08 PMI was monitoring memory usage, and it used about 8-9GB memory, so around half of the xlarge
jonty
02/16/2022, 9:09 PMAbout 55m records in total, so not huge at all, and we don't have particularly wide tables. I'm surprised it used that memory, but I guess that's java for you
jonty
02/16/2022, 9:09 PMI wonder if t3.large will be enough, or if that will fall over as well
5 Views