Hello, I am trying to use the Airbyte API in a scr...
# help-api-cli-orchestration
t
Hello, I am trying to use the Airbyte API in a script to to create 4 different connections between 4 postgres sources and the same bigquery destination. Once I was able to successfully create 4 connections. But I usually get one of these errors:
data: 'stream timeout'
or
data: 'upstream request timeout'
I believe the create connection api call is failing. AirByte OSS version:
0.50.30
Postgres Source version:
3.1.8
BigQuery Destination version:
2.0.16
Any ideas?
u
Based on the information provided in the extracts, it seems like you might be encountering a timeout issue. This could be due to a variety of reasons such as insufficient resources, network issues, or configuration problems. Here are a few suggestions to troubleshoot this issue: 1. Check your server configuration: The server might be timing out. You can check this by looking at the server logs or the network tab in your web browser. If you see a 504 Gateway Timeout error with the payload
upstream request timeout
, this might indicate a configuration issue with your instance. You might need to increase the timeout configured for your server. [Here](https://kitcharoenp.github.io/nginx/2021/10/21/upstream-timed-out.html) is a guide on how to do this. 2. Ensure sufficient resources: Make sure Airbyte has enough resources. You can check the scheduler logs for any issues. Here is a guide on [scaling Airbyte on Kubernetes](https://docs.airbyte.com/operator-guides/scaling-airbyte/#on-kubernetes). 3. Check your Postgres and BigQuery configurations: Make sure you're using the correct versions of Postgres and BigQuery connectors and that they're configured correctly. For Postgres, you should be using v9.3.x or above for non-CDC workflows and Postgres v10 or above for CDC workflows. For BigQuery, ensure SSL is enabled in your environment if you're using Airbyte Cloud. Here is the [Postgres troubleshooting guide](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting) for more information. 4. Check your connection configurations: When creating a connection in Airbyte, you need to select the correct source and destination, set the replication frequency, specify the namespace, and configure the streams and sync mode. Here is a [tutorial on creating a connection from Postgres to BigQuery](https://airbyte.com/tutorials/postgres-to-bigquery). If none of these suggestions work, it might be helpful to file a GitHub issue with the details of your problem so the Airbyte team can help you further.
t
I still get a 504 gateway timeout with
data: 'stream timeout'
or
data: 'upstream request timeout'
response even though proxy timeout set to 1800 seconds?