<@U04S8PLJWMT> the airbyte-worker throw this error...
# ask-ail
Luis Gustavo Macedo Lousada
12/08/2025, 8:19 PM@kapa.ai the airbyte-worker throw this error when try test a destination, how can fix or debug this bro???
Copy code
2025-12-08 19:34:30,213 [Workflow Executor taskQueue="ui_commands", namespace="default": 1] WARN i.t.i.r.ReplayWorkflowTaskHandler(failureToWFTResult):302 - Workflow task processing failure. startedEventId=3, WorkflowId=check_b83dd098-8e2d-4741-84f5-0bd4390fd5f8, RunId=019aff75-4b49-7898-aff4-ecf3f6c52233. If seen continuously the workflow might be stuck.
io.temporal.internal.statemachines.InternalWorkflowTaskException: Failure handling event 3 of type 'EVENT_TYPE_WORKFLOW_TASK_STARTED' during execution. {WorkflowTaskStartedEventId=3, CurrentStartedEventId=3} Copy code
Caused by: io.temporal.internal.sync.PotentialDeadlockException: [TMPRL1101] Potential deadlock detected. Workflow thread "workflow-method-check_b83dd098-8e2d-4741-84f5-0bd4390fd5f8-019aff75-4b49-7898-aff4-ecf3f6c52233" didn't yield control for over a second. {detectionTimestamp=1765222469954, threadDumpTimestamp=1765222469957}
workflow-method-check_b83dd098-8e2d-4741-84f5-0bd4390fd5f8-019aff75-4b49-7898-aff4-ecf3f6c52233
at kotlin.reflect.jvm.internal.impl.types.KotlinTypeFactory.flexibleType(KotlinTypeFactory.kt:188)k
kapa.ai
12/08/2025, 8:20 PMYour error is the same Temporal “potential deadlock” pattern that appears in existing Airbyte reports, just with a different stack frame (Kotlin lib instead of
JarFileWorkflow task processing failure … If seen continuously the workflow might be stuck.io.temporal.internal.sync.PotentialDeadlockException: [TMPRL1101] Potential deadlock detected. Workflow thread "workflow-method-check_…" didn't yield control for over a second.WorkerException: Error while getting checking connectionTimed out waiting for [300000] milliseconds for [Pod] with name:[destination-...-check-...]destination-<connector>-check-...Pendingairbyte-workerPotentialDeadlockExceptionairbyte-workerl
Luis Gustavo Macedo Lousada
12/08/2025, 9:52 PM@kapa.ai This is full stack trace
2025-12-08 193430,213 [Workflow Executor taskQueue="ui_commands", namespace="default": 1] WARN i.t.i.r.ReplayWorkflowTaskHandler(failureToWFTResult):302 - Workflow task processing failure. startedEventId=3, WorkflowId=check_b83dd098-8e2d-4741-84f5-0bd4390fd5f8, RunId=019aff75-4b49-7898-aff4-ecf3f6c52233. If seen continuously the workflow might be stuck.
io.temporal.internal.statemachines.InternalWorkflowTaskException: Failure handling event 3 of type 'EVENT_TYPE_WORKFLOW_TASK_STARTED' during execution. {WorkflowTaskStartedEventId=3, CurrentStartedEventId=3}
at io.temporal.internal.statemachines.WorkflowStateMachines.createEventProcessingException(WorkflowStateMachines.java:436)
at io.temporal.internal.statemachines.WorkflowStateMachines.handleEventsBatch(WorkflowStateMachines.java:338)
at io.temporal.internal.statemachines.WorkflowStateMachines.handleEvent(WorkflowStateMachines.java:297)
at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.applyServerHistory(ReplayWorkflowRunTaskHandler.java:260)
at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.handleWorkflowTaskImpl(ReplayWorkflowRunTaskHandler.java:242)
at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.handleWorkflowTask(ReplayWorkflowRunTaskHandler.java:165)
at io.temporal.internal.replay.ReplayWorkflowTaskHandler.handleWorkflowTaskWithQuery(ReplayWorkflowTaskHandler.java:135)
at io.temporal.internal.replay.ReplayWorkflowTaskHandler.handleWorkflowTask(ReplayWorkflowTaskHandler.java:100)
at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handleTask(WorkflowWorker.java:475)
at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:366)
at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:306)
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$1(PollTaskExecutor.java:96)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.RuntimeException: WorkflowTask: failure executing SCHEDULED->WORKFLOW_TASK_STARTED, transition history is [CREATED->WORKFLOW_TASK_SCHEDULED]
at io.temporal.internal.statemachines.StateMachine.executeTransition(StateMachine.java:163)
at io.temporal.internal.statemachines.StateMachine.handleHistoryEvent(StateMachine.java:103)
at io.temporal.internal.statemachines.EntityStateMachineBase.handleEvent(EntityStateMachineBase.java:84)
at io.temporal.internal.statemachines.EntityStateMachineInitialCommand.handleEvent(EntityStateMachineInitialCommand.java:70)
at io.temporal.internal.statemachines.WorkflowStateMachines.handleSingleEvent(WorkflowStateMachines.java:487)
at io.temporal.internal.statemachines.WorkflowStateMachines.handleEventsBatch(WorkflowStateMachines.java:336)
... 13 common frames omitted
Caused by: io.temporal.internal.sync.PotentialDeadlockException: [TMPRL1101] Potential deadlock detected. Workflow thread "workflow-method-check_b83dd098-8e2d-4741-84f5-0bd4390fd5f8-019aff75-4b49-7898-aff4-ecf3f6c52233" didn't yield control for over a second. {detectionTimestamp=1765222469954, threadDumpTimestamp=1765222469957}
workflow-method-check_b83dd098-8e2d-4741-84f5-0bd4390fd5f8-019aff75-4b49-7898-aff4-ecf3f6c52233
at kotlin.reflect.jvm.internal.impl.types.KotlinTypeFactory.flexibleType(KotlinTypeFactory.kt:188)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.types.JavaTypeResolver.transformJavaClassifierType(JavaTypeResolver.kt:117)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.types.JavaTypeResolver.transformJavaType(JavaTypeResolver.kt:58)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.types.JavaTypeResolver.transformToTypeProjection(JavaTypeResolver.kt:305)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.types.JavaTypeResolver.computeArguments(JavaTypeResolver.kt:275)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.types.JavaTypeResolver.computeSimpleJavaClassifierType(JavaTypeResolver.kt:135)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.types.JavaTypeResolver.transformJavaClassifierType(JavaTypeResolver.kt:104)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.types.JavaTypeResolver.transformJavaType(JavaTypeResolver.kt:58)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaClassDescriptor$LazyJavaClassTypeConstructor.computeSupertypes(LazyJavaClassDescriptor.kt:226)
at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$1.invoke(AbstractTypeConstructor.kt:78)
at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$1.invoke(AbstractTypeConstructor.kt:77)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:408)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValueWithPostCompute.invoke(LockBasedStorageManager.java:481)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValueWithPostCompute.invoke(LockBasedStorageManager.java:512)
at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor.getSupertypes(AbstractTypeConstructor.kt:27)
at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor.getSupertypes(AbstractTypeConstructor.kt:26)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaClassMemberScope.computeFunctionNames(LazyJavaClassMemberScope.kt:74)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaClassMemberScope.computeFunctionNames(LazyJavaClassMemberScope.kt:63)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$functionNamesLazy$2.invoke(LazyJavaScope.kt:259)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$functionNamesLazy$2.invoke(LazyJavaScope.kt:259)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:408)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:527)
at kotlin.reflect.jvm.internal.impl.storage.StorageKt.getValue(storage.kt:42)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.getFunctionNamesLazy(LazyJavaScope.kt:259)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.getFunctionNames(LazyJavaScope.kt:263)
at kotlin.reflect.jvm.internal.impl.builtins.jvm.JvmBuiltInsCustomizer.getFunctionsNames(JvmBuiltInsCustomizer.kt:184)
at kotlin.reflect.jvm.internal.impl.builtins.jvm.JvmBuiltInsCustomizer.getFunctionsNames(JvmBuiltInsCustomizer.kt:50)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope.getNonDeclaredFunctionNames(DeserializedClassDescriptor.kt:354)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$OptimizedImplementation$functionNames$2.invoke(DeserializedMemberScope.kt:258)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$OptimizedImplementation$functionNames$2.invoke(DeserializedMemberScope.kt:257)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:408)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:527)
at kotlin.reflect.jvm.internal.impl.storage.StorageKt.getValue(storage.kt:42)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$OptimizedImplementation.getFunctionNames(DeserializedMemberScope.kt:257)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope.getFunctionNames(DeserializedMemberScope.kt:59)
at kotlin.reflect.jvm.internal.impl.resolve.scopes.SubstitutingScope.getFunctionNames(SubstitutingScope.kt:89)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope.getNonDeclaredFunctionNames(DeserializedClassDescriptor.kt:353)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$OptimizedImplementation$functionNames$2.invoke(DeserializedMemberScope.kt:258)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$OptimizedImplementation$functionNames$2.invoke(DeserializedMemberScope.kt:257)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:408)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:527)
at kotlin.reflect.jvm.internal.impl.storage.StorageKt.getValue(storage.kt:42)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$OptimizedImplementation.getFunctionNames(DeserializedMemberScope.kt:257)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$OptimizedImplementation.addFunctionsAndPropertiesTo(DeserializedMemberScope.kt:357)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope.computeDescriptors(DeserializedMemberScope.kt:115)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope$allDescriptors$1.invoke(DeserializedClassDescriptor.kt:277)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope$allDescriptors$1.invoke(DeserializedClassDescriptor.kt:276)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:408)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:527)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope.getContributedDescriptors(DeserializedClassDescriptor.kt:287)
at kotlin.reflect.jvm.internal.impl.resolve.scopes.InnerClassesScopeWrapper.getContributedDescriptors(InnerClassesScopeWrapper.kt:35)
at kotlin.reflect.jvm.internal.impl.resolve.scopes.InnerClassesScopeWrapper.getContributedDescriptors(InnerClassesScopeWrapper.kt:27)
at kotlin.reflect.jvm.internal.impl.resolve.scopes.ResolutionScope$DefaultImpls.getContributedDescriptors$default(ResolutionScope.kt:50)
at kotlin.reflect.jvm.internal.KClassImpl$Data$nestedClasses$2.invoke(KClassImpl.kt:111)
at kotlin.reflect.jvm.internal.KClassImpl$Data$nestedClasses$2.invoke(KClassImpl.kt:110)
at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:70)
at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:32)
at kotlin.reflect.jvm.internal.KClassImpl$Data.getNestedClasses(KClassImpl.kt:110)
... more
k
kapa.ai
12/08/2025, 9:53 PMA message from kapa.ai
kapa.ai
12/08/2025, 9:54 PMThis stack trace matches the Temporal “potential deadlock” pattern already reported for Airbyte checks, just with a different Java/Kotlin stack. [[Temporal deadline on EKS](https://github.com/airbytehq/airbyte/issues/59730)] Key points from your log: -
ReplayWorkflowTaskHandler … Workflow task processing failure … If seen continuously the workflow might be stuck.Caused by: java.lang.RuntimeException: WorkflowTask: failure executing SCHEDULED->WORKFLOW_TASK_STARTED…Caused by: io.temporal.internal.sync.PotentialDeadlockException: [TMPRL1101] Potential deadlock detected. Workflow thread "workflow-method-check_…" didn't yield control for over a second.io.airbyte.workers.exception.WorkerException: Error while getting checking connection.Caused by: io.fabric8.kubernetes.client.KubernetesClientTimeoutException: Timed out waiting for [300000] milliseconds for [Pod] with name:[destination-...-check-...] in namespace [default].airbyte-worker2025-12-08 19:34:30WorkerExceptiondestination-<name>-check-<id>-<rand>Running Copy code
bash kubectl get pods -n <airbyte-namespace> | grep check_b83dd098 kubectl get pods -n <airbyte-namespace> | grep destination- # or your destination typeInit:0/1kubectl describe pod ...DEADLINE_EXCEEDED …airbyte-workerairbyte-temporalairbyte-workerWorkerExceptiondestination-...-checkDefaultCheckConnectionWorkerAirbyteIntegrationLauncherKubeProcessFactoryKubePodProcessl
Luis Gustavo Macedo Lousada
12/08/2025, 9:59 PM@kapa.ai check pods not is created
k
kapa.ai
12/08/2025, 9:59 PMA message from kapa.ai
kapa.ai
12/08/2025, 9:59 PMA message from kapa.ai
kapa.ai
12/08/2025, 10:00 PMIf the check pod is not created at all, you’re in the same situation described in the Temporal / EKS issue and some K8s threads: > “the check pod never starts, and the Kubernetes cluster does not show any logs indicating that the pod was requested to be created. I also noticed that the airbyte-worker seems to be stuck after launching…” [[Temporal deadline on EKS](https://github.com/airbytehq/airbyte/issues/59730)] In those cases: - Temporal logs show
PotentialDeadlockExceptioncheck_...destination-...-check-...KubeClientExceptionKubernetesClientExceptionairbyte-workerKubeClientException: Failed to create pod ...KubernetesClientExceptionKubernetesClientTimeoutExceptionjava.io.FileNotFoundException: /root/.kube/configairbyte-workerRunningAttempting to start pod = destination-...-check-...KubeProcessFactoryKubePodClientWorkerException: Error while getting checking connectionWorkerExceptionKubeProcessFactory Copy code
bash kubectl logs -n <airbyte-namespace> deploy/airbyte-worker -fcheck_b83dd098...DefaultCheckConnectionWorkerAirbyteIntegrationLauncher.checkKubeProcessFactory.createKubePodClient.launchCheckAttempting to start pod = ...kapa.ai
12/08/2025, 10:00 PMIf the check pods are not created at all, Airbyte’s worker/Temporal are failing before or while trying to ask Kubernetes to create them. The behavior you describe matches existing reports where: - The user triggers test connection, but: - No
*-check-*airbyte-worker Copy code
text Attempting to start pod = destination-s3-check-192-0-qcbyl for airbyte/destination-s3:... ... Creating pod destination-s3-check-192-0-qcbyl...source-...-check-...Attempting to start pod = ...check...WorkerExceptionDEADLINE_EXCEEDED Copy code
text WorkerException: Operation: [create] for kind: [Pod] ... failed. Caused by: io.fabric8.kubernetes.client.KubernetesClientException Copy code
text Timed out waiting for [300000] milliseconds for [Pod] with name:[destination-snowflake-check-...] in namespace [default]. Caused by: KubernetesClientTimeoutExceptionairbyte-workerOperation: [create] for kind: [Pod]KubernetesClientExceptionKubernetesClientTimeoutExceptionTimed out waiting for [300000] milliseconds for [Pod]airbyte-worker~/.kube/configKUBECONFIGjava.io.FileNotFoundException: /root/.kube/configkubectl describe pod ...PotentialDeadlockExceptioncheck_*airbyte-workerairbyte-temporalairbyte-workerAttempting to start podkubectl logs <airbyte-worker-pod> -n <ns>Attempting to start pod =Operation: [create] for kind: [Pod]KubernetesClientExceptionKubernetesClientTimeoutExceptionkubectl get pods -n <ns> | grep check-airbyte-workerDefaultCheckConnectionWorkerAirbyteIntegrationLauncherKubeProcessFactory