Hi <@U04S8PLJWMT>, all my connection are failing w...
# ask-ai
a
Hi @kapa.ai, all my connection are failing with Airbyte Platform is experiencing a higher than usual load, please try again later.
k
This message indicates a platform/backend problem rather than a misconfigured connector. The shared sources show similar symptoms (very long UI/API calls, then HTTP 502 or generic “temporarily unavailable” errors) caused by overloaded or unhealthy Airbyte services, often due to: - High or leaking memory usage in core services (server/worker), requiring restarts or upgrades. [[Long loading time](https://discuss.airbyte.io/t/2536)] - Backend API processes (abctl worker / workload API) needing a rollout restart in Kubernetes. [[ABCTL restart fix](https://github.com/airbytehq/airbyte/issues/66692)] Given what’s documented, the concrete actions you can try are: 1. Restart the core backend services - On Kubernetes with `abctl`:
Copy code
bash      kubectl -n airbyte-abctl rollout restart deployment airbyte-abctl-worker      kubectl -n airbyte-abctl rollout restart deployment airbyte-abctl-workload-api-server       kubectl -n airbyte-abctl rollout status deployment airbyte-abctl-worker      kubectl -n airbyte-abctl rollout status deployment airbyte-abctl-workload-api-server
This resolved 502 / “temporarily unavailable” errors for other users. [[ABCTL restart fix](https://github.com/airbytehq/airbyte/issues/66692)] - On classic OSS Docker: restart
airbyte-server
and
airbyte-worker
containers, which is the suggested workaround when UI/API calls become extremely slow or start returning 502. [[Long loading time](https://discuss.airbyte.io/t/2536)] 2. Check resource usage Inspect CPU and memory of Airbyte containers or pods (e.g.
docker stats
or
kubectl top pods
) to confirm if they’re close to limits; prior reports of 502/slow UI were tied to high memory usage and logging-related leaks. [[Long loading time](https://discuss.airbyte.io/t/2536)] 3. Upgrade Airbyte if you’re on an older version The team has shipped optimizations and fixes to reduce backend memory usage and improve DB queries; upgrading was explicitly recommended as a mitigation for slow connection pages and 502s. [[Long loading time](https://discuss.airbyte.io/t/2536); [Error after upgrading](https://discuss.airbyte.io/t/2537)] If, after restarting the backend services and verifying resources, all connections still immediately fail with the same “higher than usual load” message, the knowledge sources do not provide more specific remediation steps. In that case, you’ll likely need to: - Capture server/worker (or abctl) logs around a failing attempt, and - Open a GitHub issue or forum topic with those logs and your deployment details (version, platform, resource limits), as other users did in the linked threads.
a
The airbyte instance is deployed on compute engine VM on google with an external postgres on another
@kapa.ai
k
Sorry @Andrea Brenna, I'm not sure what you mean. Please try again. 👋
a
@kapa.ai The airbyte instance is deployed on compute engine VM on google with an external postgres on another
k
A message from kapa.ai
Given your earlier symptom (“Airbyte Platform is experiencing a higher than usual load…”) and that you’re running: - Airbyte on a GCE VM, and - An external Postgres on another instance (or managed service), the most relevant issues in the knowledge sources around external Postgres backends point to connectivity, Postgres limits, or firewall/VPC problems rather than generic “load” in a strict sense. Here are focused checks you can run, based strictly on the docs and forum posts: --- ### 1. Verify network connectivity from the Airbyte VM Multiple users had problems with external Postgres (RDS / Cloud SQL) where Airbyte could partially connect (create some tables) but then continually failed. Root causes included: - Firewall rules blocking or dropping packets between the Airbyte host and Postgres. Fixing firewall rules fully resolved the deployment and connection failures in one RDS case. [[RDS firewall fix](https://discuss.airbyte.io/t/2179)] - VPC / network assumptions: support explicitly asked users to confirm that Airbyte and the DB were in the same network / reachable from containers, and to test with
psql
from inside a container. [[Failed external DB](https://discuss.airbyte.io/t/1264)] Concrete actions: 1. From the Airbyte VM shell, run
psql
against your external Postgres and confirm you can: - Connect, - Run simple queries, - Leave it running for a while to ensure no intermittent drops. 2. If you’re using Docker, also exec into a core container (e.g.
airbyte-server
) and run
psql
from there, to confirm container-level connectivity. If
psql
intermittently fails or hangs, inspect and fix any firewall rules or security groups between the VM and the DB. [[RDS firewall fix](https://discuss.airbyte.io/t/2179)] --- ### 2. Check Postgres connection and resource limits A documented case on GCP Cloud SQL (external Postgres) shows Airbyte could hit the DB, create tables, but the server then hung during startup with repeated “Waiting for database to become available…” messages. Root cause was **exhausted Postgres connections**; it was fixed by: - Lowering
SQL_MAX_CONNS
and
SQL_MAX_IDLE_CONNS
on the Airbyte side, and - Increasing
max_connections
on the Postgres instance. [[Cloud SQL limits](https://discuss.airbyte.io/t/554)] If your Airbyte instance is under some load (many connections or jobs) and uses an external DB with relatively low
max_connections
, you can experience: - DB connection pool exhaustion, - Timeouts or “cannot reach server / higher than usual load” at the API. Actions: 1. On the Postgres instance, check current connections vs
max_connections
. 2. If near the limit, either: - Increase Postgres
max_connections
, and/or - Reduce Airbyte DB connection settings (where available; for older setups this was done via `SQL_MAX_CONNS`/`SQL_MAX_IDLE_CONNS` env vars). [[Cloud SQL limits](https://discuss.airbyte.io/t/554)] --- ### 3. Validate external-DB configuration matches the docs For Enterprise / Helm deployments, the docs are explicit on how to configure an external Postgres (RDS, Cloud SQL, etc.): - Disable the internal
airbyte/db
Postgres. - Set
global.database
host, port, db name, and credentials in
values.yaml
, optionally through
airbyte-config-secrets
. [[Enterprise DB config](https://docs.airbyte.com/platform/enterprise-setup/implementation-guide#configuring-the-airbyte-database)] Example (Helm v1 pattern):
Copy code
yaml postgresql:   enabled: false  global:   database:     secretName: "airbyte-config-secrets"     host: "YOUR_DB_HOST"     port: "5432"     database: "YOUR_DB_NAME"     user: "YOUR_DB_USER"     passwordSecretKey: "database-password"
If any of these values are wrong, Airbyte will sit in a loop of “database not ready / retrying” and services will never fully become healthy, which then surfaces as UI/API errors. Since you’re on a GCE VM, you may instead be using Docker and
.env
variables as shown in several topics:
Copy code
env DATABASE_USER=... DATABASE_PASSWORD=... DATABASE_HOST=... DATABASE_PORT=5432 DATABASE_DB=... DATABASE_URL=jdbc:<postgresql://HOST:5432/DBNAME>
Misconfiguration there (wrong host, port, SSL parameters) caused “Database is not ready yet” loops for other users trying to connect from a VM to local or external Postgres. [[Unable to connect external DB](https://discuss.airbyte.io/t/729); [Cloud SQL limits](https://discuss.airbyte.io/t/554)] Action: - Double-check
.env
(for Docker) or `values.yaml`/secrets (for Helm) against the documented patterns and against a working
psql
connection string. --- ### 4. Watch both Airbyte and Postgres logs during a failing attempt The issues above were debugged by watching: -
airbyte-server
and
airbyte-worker
(or scheduler) logs: - Look for “Waiting for database to become available…”, “failed to verify database connection”, or repeated connection retries. [[Cloud SQL limits](https://discuss.airbyte.io/t/554); [Unable to connect external DB](https://discuss.airbyte.io/t/729); [Failed external DB](https://discuss.airbyte.io/t/1264)] - Postgres logs: - Connection refused / authentication failures, - “too many connections”, - Timeouts or dropped connections. This will indicate whether your “higher than usual load” message is really: - DB connection pool exhaustion, - Network/firewall packet drops, - Or a misconfig in the external DB settings. --- ### What the knowledge does not cover The sources don’t contain an explicit description of the exact “Airbyte Platform is experiencing a higher than usual load” UI message or a dedicated troubleshooting page for it. They instead document concrete root causes (firewall, max connections, misconfig) in external Postgres setups that lead to similar symptoms (server not starting, UI cannot reach server, etc.). [[RDS firewall fix](https://discuss.airbyte.io/t/2179); [Cloud SQL limits](https://discuss.airbyte.io/t/554); [Unable to connect external DB](https://discuss.airbyte.io/t/729); [Failed external DB](https://discuss.airbyte.io/t/1264)] If, after: - Confirming stable
psql
connectivity from both VM and containers, - Ensuring Postgres
max_connections
is not being hit, - Verifying your external DB configuration, you still see all connections failing with that same platform load message, the knowledge sources don’t provide further, more specific steps. In that case, you should capture: -
airbyte-server
and
airbyte-worker
logs around a failing run, - Postgres logs and connection metrics, and open a new forum topic or GitHub issue referencing your external-Postgres setup and what you’ve already tried.