Ishan Anilbhai Koradiya
12/02/2025, 4:48 AMkapa.ai
12/02/2025, 4:48 AM/api/v1/source_definitions/list_enterprise_source_stubs and fails with: > Internal Server Error: java.io.IOException: HTTP error fetching enterprise sources [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609); [UI 500 error](https://github.com/airbytehq/airbyte/issues/54058)]. A temporary workaround used by OSS users was: - Host a file containing just [] (empty JSON array). - Set the ENTERPRISE_SOURCE_STUBS_URL environment variable to point to that file, so Airbyte stops trying (and failing) to fetch real enterprise stubs. [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609)] In that specific case, the underlying endpoint cache was later fixed and the issue resolved by the team. [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609); [UI 500 error](https://github.com/airbytehq/airbyte/issues/54058)] 2. Connector/underlying API returning 500 during sync: Some connectors (for example source-posthog) can start failing because the remote API endpoint begins returning 500s or timeouts (e.g., deprecated endpoints). [[PostHog 500](https://github.com/airbytehq/airbyte/issues/60293)] In those cases, the Airbyte job logs for the specific sync/check attempt will usually show the external HTTP 500 and possibly connector‑specific stack traces. Given your single log line: text createNewAttemptNumber for job 60048 failed with exception: HTTP error: 500 Internal Server Error the knowledge sources do not give a direct mapping from that exact message to a single known bug or config problem. I can’t reliably say which of the above patterns (or something else) you’re hitting without more context. How to narrow it down (based on available info): 1. Check if it’s the enterprise stubs issue: - Look in airbyte-server logs around job 60048 for any call to /api/v1/source_definitions/list_enterprise_source_stubs returning 500 with message HTTP error fetching enterprise sources. [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609); [UI 500 error](https://github.com/airbytehq/airbyte/issues/54058)] - If that’s present and you’re on an OSS/self‑hosted deployment, you can: - Temporarily set ENTERPRISE_SOURCE_STUBS_URL to a hosted [] file as described above, then restart airbyte-server. [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609)] - Or upgrade to a version where this endpoint/cache issue is already fixed (the linked issues are closed as resolved). [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609); [UI 500 error](https://github.com/airbytehq/airbyte/issues/54058)] 2. If the 500 is from a source/destination API: - Open the full job logs for job 60048 in the UI and check: - Which connector is in use. - Whether the logs show remote API 500s or timeouts similar to the PostHog case. [[PostHog 500](https://github.com/airbytehq/airbyte/issues/60293)] - If so, adjusting error‑handling on HTTP 500s may help in connector‑builder connectors, by treating 500s as retryable instead of immediate failure (via HTTP response filters). [[HTTP codes handling](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/error-handling#http-codes)] - Example idea (for config‑based/Builder connectors): add a response filter with http_codes: [500] and action: RETRY so that transient 500s are retried instead of failing the job. [[HttpResponseFilter](https://docs.airbyte.com/platform/1.8/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/HttpResponseFilter); [HTTP codes handling](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/error-handling#http-codes)] Because the sources you provided don’t include your exact deployment version, connector name, or the surrounding stack trace, I can’t be more specific than the above. If you can share: - The Airbyte version and deployment type (Docker, Kubernetes, abctl, etc.). - The connector and action (test connection vs. sync). - A larger excerpt of the server/job logs around job 60048. then I can map it more directly to the closest documented issue or configuration option from the knowledge sources.Ishan Anilbhai Koradiya
12/02/2025, 4:56 AMkapa.ai
12/02/2025, 4:56 AMkapa.ai
12/02/2025, 4:57 AMCreateNewAttemptNumber is calling the Airbyte Server Attempt API and receiving HTTP 500. - The error is raised by ThrowOn5xxInterceptor inside the Airbyte API client, so the server itself is returning 500 for AttemptApi.createNewAttemptNumber. [[v1.6.0 deploy 5xx](https://github.com/airbytehq/airbyte/issues/58587)] - OkHttp warns about “connection leaked… did you forget to close a response body?” for http://airbyte-airbyte-server-svc:8001/, which indicates client connections to the server are not being closed cleanly, potentially under heavy/failed calls. From the knowledge you provided, there are similar patterns: - Internal HTTP 500s from Airbyte server (e.g. when fetching enterprise connector stubs, or from cache issues) manifested as generic 500 errors in the UI/API and were ultimately solved by fixing the server bug or restarting/upgrading the server. [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609); [Unexpected 500 cache](https://github.com/airbytehq/airbyte/issues/54058)] - A 5xx interceptor in the abctl / platform client surfaces server‐side 500 errors in the same way you see in your logs. [[v1.6.0 deploy 5xx](https://github.com/airbytehq/airbyte/issues/58587)] However, the sources you shared do not contain: - The implementation details of AttemptApi.createNewAttemptNumber. - A known bug specifically about JobCreationAndStatusUpdateActivityImpl.createNewAttemptNumber returning 500. - A documented fix or configuration for the “connection leaked” warnings in this context. So I can’t point to a definitive root cause or a version‑specific bug for this exact stack trace from the knowledge set. Based on what is documented, the most relevant, low‑risk steps are: 1. Inspect the Airbyte Server logs around the 500s Look at the airbyte-server pod logs at the timestamps matching these Temporal activity errors. In similar cases, the server logs clearly showed the deeper reason (e.g. failing to fetch enterprise stubs, cache inconsistency). [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609); [Unexpected 500 cache](https://github.com/airbytehq/airbyte/issues/54058)] 2. Restart airbyte-server For at least one internal 500 issue, maintainers indicated it was related to a cache issue and suggested restarting the airbyte-server to clear the problem. [[Unexpected 500 cache](https://github.com/airbytehq/airbyte/issues/54058)] If you haven’t already, restart only the airbyte-server deployment/pod and see if the CreateNewAttemptNumber calls start succeeding. 3. Check/upgrade to a version where similar 500s are marked as fixed Some 500s (e.g. enterprise stub fetching) were resolved in subsequent releases and the corresponding issues are closed. [[HTTP 500 enterprise](https://github.com/airbytehq/airbyte/issues/53609); [Unexpected 500 cache](https://github.com/airbytehq/airbyte/issues/54058)] If you’re on an older version than those fixes, upgrading Airbyte (and redeploying cleanly) is aligned with what support recommended in similar threads. [[K8s 502/health](https://discuss.airbyte.io/t/594); [K8s services errors](https://discuss.airbyte.io/t/2861)] 4. Verify deployment health (Kubernetes) In other Kubernetes deployments where UI/API operations failed (502/500s, Temporal workflow errors), the root issue was an unhealthy deployment (Temporal or server not reachable, pods spread across nodes causing volume/log issues, etc.). [[K8s 502/health](https://discuss.airbyte.io/t/594); [Gateway timeout / logs missing](https://discuss.airbyte.io/t/376)] Basic checks you can do, consistent with those threads: - All core pods (airbyte-server, airbyte-worker, Temporal, DB) are Running and ready. - Temporal is reachable from worker pods (no Connection refused to Temporal). [[K8s 502/health](https://discuss.airbyte.io/t/594)] - No obvious out‑of‑memory or crashloop patterns in server/worker logs. Because the knowledge sources don’t describe this exact CreateNewAttemptNumber 500 or the OkHttp leak warnings as a known issue with a documented fix, I can’t go beyond these general, evidence‑backed steps. If you can pull and share (in your environment or with Airbyte support) the airbyte-server logs corresponding to one of these 500s, that should reveal the real server‑side exception, which is the next necessary step to resolve it.