Hello! Is there any documentation on how different...
# feedback-and-requests
o
Hello! Is there any documentation on how different configurations of BQ Destination would impact BQ query quota usage? Thanks!
Transformation Priority: configure the priority of queries run for transformations. Refer to https://cloud.google.com/bigquery/docs/running-queries. By default, Airbyte runs interactive query jobs on BigQuery, which means that the query is executed as soon as possible and count towards daily concurrent quotas and limits. If set to use batch query on your behalf, BigQuery starts the query as soon as idle resources are available in the BigQuery shared resource pool. This usually occurs within a few minutes. If BigQuery hasn’t started the query within 24 hours, BigQuery changes the job priority to interactive. Batch queries don’t count towards your concurrent rate limit, which can make it easier to start many queries at once.
r
Thanks for the info @Chris (deprecated profile). I was more referring to the billableBytes caused by BQ connector in incremental syncs, I noticed they are proportional to the size of the whole dataset (not the size of the incremental batch). I think they are mainly a result of the normalization dbt jobs at the end. However I tested the Denormalized version of BQ connector and it doesn't induce high amount of BillableBytes. Many thanks