For timeout errors we ve upped the read timeout time in this Airbyte #airbyte-api

For timeout errors, we’ve upped the read-timeout t...

Terence Cho (Airbyte)

09/05/2023, 5:48 PM

For timeout errors, we’ve upped the read-timeout time in this commit. which was released in version

v0.50.25

. And also edited the error code to

instead of

for create jobs endpoint, but we’ll be adding it to all the relevant endpoints. We’re still looking into causes for this issue, one of the known causes are when there are interactions needed with an external resource outside of airbyte (such as a connector) that takes longer to respond. We will continue to look into these issues to reduce the number of timeouts and

errors.

Nina Jensen

09/06/2023, 8:27 AM

Thanks for the update. Still facing issues on the new version - the API server simply crashes now when I try to reach endpoints, on k8s gcp deployed with Helm (dev instance).

10.76.1.18

is the API server pod IP - it's my understanding that the API server simply federates calls to the "old" config API, logs seem to confirm and response looks fine:

Copy code

2023-09-06 08:23:55 INFO i.m.h.s.n.h.a.e.AccessLog(log):125 - 10.76.1.18 - - [06/Sep/2023:08:23:55 +0000] "POST /api/v1/workspaces/list │
│  HTTP/1.1" 200 850

However this seemingly never reaches the API pod, which will sometimes throw a 500, most times simply crash.

Nina Jensen

09/06/2023, 8:40 AM

And here are the API server pod logs, it's my understanding that we should see a

localhost

entry for

api/v1/workspaces

but it crashes:

Copy code

2023-09-06 08:23:17 INFO i.m.c.e.DefaultEnvironment(<init>):167 - Established active environments: [k8s, cloud, gcp]
2023-09-06 08:23:21 INFO i.m.r.Micronaut(lambda$start$2):98 - Startup completed in 6806ms. Server Running: <http://airbyte-dev-ingest-airbyte-api-server-f984df9c9-tdffg:8006>
2023-09-06 08:23:35 INFO i.m.h.s.n.h.a.e.AccessLog(log):125 - localhost - - [06/Sep/2023:08:23:35 +0000] "GET /health HTTP/1.1" 200 20
2023-09-06 08:26:18 INFO i.m.r.Micronaut(lambda$start$0):115 - Embedded Application shutting down

(filtered out all other calls to /health)

Nina Jensen

09/07/2023, 1:17 PM

Never mind this, just tried to deploy Airbyte locally and the API works as expected, so must be something related to our setup.

Julien Kernec'h

10/09/2023, 10:18 PM

I face the same issue on a fresh K8s deployment on GCP, aside from the health endpoint, every single endpoints is crashing the api

Julien Kernec'h

10/09/2023, 10:20 PM

@Terence Cho (Airbyte) @Nina Jensen by any chance, did you figure out the issue or a workaround? Thanks!

Open in Slack

Previous Next