https://linen.dev logo
Join Slack
Powered by
*Is this your first time deploying Airbyte*: Non *...
# troubleshooting
j
Is this your first time deploying Airbyte: Non OS Version / Instance: Ubuntu Deployment: Kubernetes Airbyte Version: 0.35.57-alpha Source name/version: Salesforce 1.0.2 Destination name/version:BigQuery Desnormalized 0.2.11 Step: When running multiple connections in different pods the worker it is not able to sync the logs between sync and destination and pods end up crashing
This is the error log I am getting
Copy code
2022-03-29 16:08:08 ERROR i.a.w.DefaultReplicationWorker(run):169 - Sync worker failed.
io.airbyte.workers.WorkerException: Operation: [create]  for kind: [Pod]  with name: [destination-bigquery-denormalized-sync-98-2-nhslv]  in namespace: [airbyter]  failed.
    at io.airbyte.workers.process.KubeProcessFactory.create(KubeProcessFactory.java:150) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at io.airbyte.workers.process.AirbyteIntegrationLauncher.write(AirbyteIntegrationLauncher.java:144) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at io.airbyte.workers.protocols.airbyte.DefaultAirbyteDestination.start(DefaultAirbyteDestination.java:70) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:127) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [create]  for kind: [Pod]  with name: [destination-bigquery-denormalized-sync-98-2-nhslv]  in namespace: [airbyter]  failed.
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:373) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.createOrReplace(CreateOrReplaceHelper.java:53) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:411) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:86) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:394) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:86) ~[kubernetes-client-5.3.1.jar:?]
    at io.airbyte.workers.process.KubePodProcess.<init>(KubePodProcess.java:509) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at io.airbyte.workers.process.KubeProcessFactory.create(KubeProcessFactory.java:146) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    ... 6 more
Caused by: java.io.FileNotFoundException: /root/.kube/config (No such file or directory)
    at java.io.FileInputStream.open0(Native Method) ~[?:?]
    at java.io.FileInputStream.open(FileInputStream.java:216) ~[?:?]
    at java.io.FileInputStream.<init>(FileInputStream.java:157) ~[?:?]
    at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:354) ~[jackson-dataformat-yaml-2.13.1.jar:2.13.1]
    at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:15) ~[jackson-dataformat-yaml-2.13.1.jar:2.13.1]
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3494) ~[jackson-databind-2.13.1.jar:2.13.1]
    at io.fabric8.kubernetes.client.internal.KubeConfigUtils.parseConfig(KubeConfigUtils.java:42) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.utils.TokenRefreshInterceptor.intercept(TokenRefreshInterceptor.java:44) ~[kubernetes-client-5.3.1.jar:?]
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.12.jar:?]
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.12.jar:?]
    at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68) ~[kubernetes-client-5.3.1.jar:?]
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.12.jar:?]
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.12.jar:?]
    at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createApplicableInterceptors$6(HttpClientUtils.java:284) ~[kubernetes-client-5.3.1.jar:?]
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[okhttp-3.12.12.jar:?]
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[okhttp-3.12.12.jar:?]
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257) ~[okhttp-3.12.12.jar:?]
    at okhttp3.RealCall.execute(RealCall.java:93) ~[okhttp-3.12.12.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:485) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:448) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:263) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:870) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:365) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.createOrReplace(CreateOrReplaceHelper.java:53) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:411) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:86) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:394) ~[kubernetes-client-5.3.1.jar:?]
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:86) ~[kubernetes-client-5.3.1.jar:?]
    at io.airbyte.workers.process.KubePodProcess.<init>(KubePodProcess.java:509) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    at io.airbyte.workers.process.KubeProcessFactory.create(KubeProcessFactory.java:146) ~[io.airbyte-airbyte-workers-0.35.57-alpha.jar:?]
    ... 6 more
I wonder why this failure happens
h
Hey how did you deploy over k8s? Also is this on GCP, Aws or something else?
2 Views
Open in Slack
PreviousNext
Powered by