Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

I am facing an issue syncing from bigquery source to redshift with the error the response is too large to return

how can I solve this, below is the error code

`Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden`
GET <https://www.googleapis.com/bigquery/v2/projects/physics-wallah-65ada/queries/108b94f5-42bd-46b1-860b-237a322eee6f?location=asia-south2&amp;maxResults=0&amp;prettyPrint=false>
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "Response too large to return. Consider specifying a destination table in your job configuration. For more details, see <https://cloud.google.com/bigquery/troubleshooting-errors>",
    "reason" : "responseTooLarge"
  } ],
  "message" : "Response too large to return. Consider specifying a destination table in your job configuration. For more details, see <https://cloud.google.com/bigquery/troubleshooting-errors>",
  "status" : "PERMISSION_DENIED"
}

Hi.
this is a limitation that exists on BQ it self.
maximum response size is 10GB or unlimited if you write it to a destination table.

you can use this link to see a workaround:

make sure to remove any order by , top and limit since this removes the parallel logic bq does on tables.
<https://cloud.google.com/bigquery/docs/writing-results#large-results>