Hello! I am running Airbyte as VM instance on GCP....
# replication-troubleshooting
s
Hello! I am running Airbyte as VM instance on GCP. Region - us-east4. And I am configuring sync from PostgreSQL to BigQuery. Using Incremental | Deduped + history mode. When I set the dataset location to us-east4. Everything works well, with no errors, and all data is in place. When I set the dataset location to US - sync fails on the normalization step. But if I set to another mode, for example, Incremental | Append. Everything works as intended. Logs in thread
✅ 1
6ed4d255_895a_4582_98f9_4b5c3da19206_logs_203_txt.txt
Airbyte - 0.40.22 Postgres connector - 1.0.27 BigQuery - 1.2.8
u
Hi, Semyon! To be perfectly honest I have not seen this type of error come up before. I'm going to ask for some ideas from my team and I'll get some feedback for you on Wednesday!
🙏 1
s
@Nataly Merezhuk (Airbyte) Thanks! I found a solution problem was in: • normalization created dataset _airbyte_api_postgres for intermediate results • after moving to another region and dropping the main schema api_postgres, and creating it once again in the new region. • _airbyte_api_postgres - still was in old region. and couldn’t be seen from anywhere. Nor in DataGrip or SQL workbench • as a fix I dropped this schema. And new normalization created new dataset in new region
u
So glad you found a solution and thanks so much for the detailed follow up!
n
Hi @Semyon Komissarov - do you mind sharing whether this schema
_airbyte_api_postgres
was in Airbyte's backend db or in the BigQuery warehouse?
s
@Nick Scheifler Hi! it was in BigQuery warehouse
👍 1
thanks 1