Hi all, quick question, is there a way to optimize...
# ask-community-for-troubleshootingm
Maykon Lopes
09/14/2022, 6:12 PMHi all, quick question, is there a way to optimize airbyte fecth data from source?
I'm currently extracting data from mysql with cdc option enabled, the table has 285GB, and the process is already running about 36 hours.
My connection is mysql source to bigquery destination (using gcs to avro)
m
Marcos Marx (Airbyte)
09/14/2022, 6:16 PMHow much memory does your Airbyte srver has?
m
Maykon Lopes
09/14/2022, 6:18 PM4 cpus 16GB of ram
m
Marcos Marx (Airbyte)
09/14/2022, 6:22 PMprobably increasing it will improve the batch size
m
Maykon Lopes
09/14/2022, 6:28 PMdo you have a tip like for extractions where initial snapshot or first full extraction table has X GB/TB your airbyte setup should be Y (cpu, mem) ?