Hey, we’re looking to use Airbyte to ETL data from...
# ask-community-for-troubleshooting
d
Hey, we’re looking to use Airbyte to ETL data from BigQuery to Postgres, but the tasks are taking a very long time. We currently have an in-house solution which extracts data to GCS and executes a COPY to load the into postgres, which takes about 10 minutes. Unfortunately, the bigquery source -> postgres destination using Airbyte is taking over 3 hours. We’ve not moving too much data, about 30M rows, about 3gb worth of data. Are there any ways of speeding this up using these connectors? Would an S3 -> postgres be faster?
s
@Duncan thanks for bringing this up. A copy might be the way to go. Could you open an issue on GitHub with this?