Hi! We are using Airbyte for data replication but...
# replication-troubleshooting
n
Hi! We are using Airbyte for data replication but facing some issues with one of our pipelines (it has a lot of data yes). I see failure logs on Airbyte saying this:
Copy code
2026-06-22 21:36:18 destination ERROR com.google.cloud.bigquery.BigQueryException: 400 Bad Request
2026-06-22 21:36:18 destination ERROR PUT <https://bigquery.googleapis.com/upload/bigquery/v2/projects/XXXXXXXXXXXXXXXXXX/jobs?uploadType=resumable&upload_id=XXXXXXXXXXXXXXXXXX>
2026-06-22 21:36:18 destination ERROR {
2026-06-22 21:36:18 destination ERROR   "error": {
2026-06-22 21:36:18 destination ERROR     "code": 400,
2026-06-22 21:36:18 destination ERROR     "message": "Request had invalid euc header.",
2026-06-22 21:36:18 destination ERROR     "errors": [
2026-06-22 21:36:18 destination ERROR       {
2026-06-22 21:36:18 destination ERROR         "message": "Request had invalid euc header.",
2026-06-22 21:36:18 destination ERROR         "domain":"**********",
2026-06-22 21:36:18 destination ERROR         "reason": "invalid"
2026-06-22 21:36:18 destination ERROR       }
2026-06-22 21:36:18 destination ERROR     ],
2026-06-22 21:36:18 destination ERROR     "status": "INVALID_ARGUMENT"
2026-06-22 21:36:18 destination ERROR   }
2026-06-22 21:36:18 destination ERROR }
Anyone else faced this before? Any workarounds?
j
We saw these for a while as well, and switching the BigQuery destination to use the Cloud Storage staging option instead of the direct writes seemed to resolve them for us.
n
Is there an alternate way? We are not aligned towards the permissions needed by the service account to write to the gcp buckets.
j
Not sure since that resolved it for us. Might be some other active community folks who have another solution or workaround. (or if you're using it on Cloud, you could try talking to support) CC: @Przemysław Dąbek @Kevin Lefevre @Fabrizio Spini
k
Hello, sorry can not help, we are using GCS as destination
🙏 1
f
Ciao @Nisheet Das, I'm using
Batched Standard Inserts
. Which is your use case? Can you report your destination configuration?
🙏 1
n
Hey @Fabrizio Spini, I'm using the standard batched insert for data replication. Probably given the large size of the initial data, the job starts breaking after a while with the above error. The same configuration has been running fine for other jobs
j
in theory the batched standard inserts are supposed to automatically split the batches to reasonable chunks (all this logic is inherited from the SDK). For a while that worked better for us than GCS staging, until this issue surfaced and I wasn't able to figure out the cause at the time but switching back to GCS staging resolved it. I tried to poke around in the source and such but never quite found a smoking gun, since things like EUC errors should be getting retried
actually, I just noticed the docs have been updated to include an entry for this: https://docs.airbyte.com/integrations/destinations/bigquery#http-400-request-had-invalid-euc-header-during-upload
oddly I don't see a setting matching "Google BigQuery Client Chunk Size" 🙃
n
Ouch. Looks like the way ahead would be to move this sync to GCS staging. Thanks a lot though!