@kapa.ai My UI for connection is not loading for a connection which has a lot of history

It’s likely that the connection UI is timing out because this connection has a large amount of data/history, and the backend call that loads its metadata is taking too long. From similar cases: • Large schemas or many tables/streams can cause the discover / load-connection API calls to exceed the default server or load balancer timeout, which then surfaces in the UI as pages that never finish loading or show 502/504 errors. [Failed schema load; Scaling timeouts] • When there are many connections or a lot of history, the

/connections/list

or equivalent backend queries can be slow enough to hit gateway/ALB idle timeouts, making the UI look empty or stuck. [Long loading time; Connections not displayed] What you can check/do: 1. Look for 502/504 or timeout errors ◦ Open your browser dev tools → Network tab and check the failing request when you open that connection (e.g.

.../web_backend/connections/get

or

.../discover_schema

). ◦ If the response mentions “upstream request timeout” or similar, it’s a timeout issue. [Failed schema load] 2. Increase server / load balancer timeouts ◦ If you’re on Kubernetes/Helm, increase Airbyte server timeouts, for example: [Scaling timeouts]

server:
         httpIdleTimeout: 20m
         extraEnv:
           - name: READ_TIMEOUT
             value: 30m

◦ If you’re behind NGINX/ALB, raise the upstream/idle timeout on the proxy or load balancer so long-running requests don’t get cut off. [Failed schema load; Connections not displayed] 3. Reduce what needs to be loaded (if possible) ◦ For very large database sources, limit the user/role to only the necessary tables/collections so discovery and metadata loading are faster. [Failed schema load] If you can share how you deploy Airbyte (Docker vs K8s, any load balancer) and what error you see in the browser network tab, that would help narrow down the exact timeout to adjust. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai how to clean up logs from DB directly?

I’m sorry, but the provided knowledge sources don’t describe any supported or safe way to “clean up logs from the DB directly” (i.e., by running SQL against Airbyte’s internal databases). They instead point to configuration-based or file/object-storage–based cleanup. From the sources, these are the documented options: 1. Workspace / job log cleanup via retention settings (newer versions) Airbyte cleans up workspace/job logs based on a retention period controlled by

TEMPORAL_HISTORY_RETENTION_IN_DAYS

. You set this as an environment variable in your deployment (Docker/Kubernetes/Helm), and Airbyte/Temporal will prune history and associated log data over time, reducing storage usage. [Log sweeping; Storage full; Hanging jobs] 2. Older (removed) workspace sweeping variables Older versions used

MINIMUM_WORKSPACE_RETENTION_DAYS

,

MAXIMUM_WORKSPACE_RETENTION_DAYS

, and

MAXIMUM_WORKSPACE_SIZE_MB

to sweep workspace files, but this mechanism was removed with the old Scheduler and replaced by the Temporal-based retention above. [Rotation logs; Log sweeping; Workspace retention vars] 3. Manual deletion of log files / object storage, not DB In some self‑hosted setups, users manually deleted log files from MinIO or the workspace volume (e.g.,

/storage/airbyte-storage/job-logging/workspace

or

/var/lib/docker/volumes/airbyte_workspace/_data

) to free space. [Hanging jobs; Quickly run out of storage; Syncs failing, disk full] 4. Direct DB manipulation is not documented / recommended Where database deletion is mentioned, it’s only in very generic terms (e.g., “identify related tables, use cascading deletes, back up first”) and explicitly warns that direct DB manipulation should be approached with caution and is not the normal way to manage connections or logs. [Builder cleanup; Disable connection via DB] Because there is no concrete, table-level guidance in the sources on which DB tables to truncate or how to safely delete log rows, I can’t provide a specific SQL procedure without guessing, which would be unsafe. If your goal is to reduce storage: • Prefer configuring

TEMPORAL_HISTORY_RETENTION_IN_DAYS

and/or cleaning up workspace/log files or MinIO objects as described above. [Log sweeping; Hanging jobs] • Avoid ad‑hoc deletes in the Airbyte DB unless you fully understand the schema and have backups; the official docs and forum posts do not provide a supported recipe for that. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.