Does anyone have any estimates on how much data is...
# feedback-and-requests
d
Does anyone have any estimates on how much data is read for egressing the data out of Google BigQuery (BQ)? We want to move data out of BQ to self hosted ClickHouse (CH) Assuming my BQ data is 1TB, will Airbyte read 1 TB or marginally over that? (Assuming I am running an incremental sync).
If you host/run the Airbyte, CH instances in the same GCP region as your GBQ tables, then you wouldn't pay for egress if you move to GCS or otherwise. Refer https://cloud.google.com/bigquery/pricing#data_extraction_pricing, but if you move between regions within a continent then there is a cost associated with it. I am assuming that you would be charged only for the queries which are being fired from Airbyte for fetching your entire data, based on the pricing model you have chosen for GBQ (say flat-rate or on-demand)
Thanks, @Rajesh Koilpillai, for these details, @Deepak Shah, is it the answer you were looking for? I confirm that Airbyte is not reading more data than the volume required for the sync, so your egress estimation is tight to your sync volume.