@kapa.ai Stage Pipeline Exception: io.airbyte.workload.launcher.pipeline.stages.model.StageError: io.airbyte.workers.exception.KubeClientException: Failed to delete pods for mutex key: 629af1e6-cc76-468b-a8cb-c17a17c778e8. message: io.airbyte.workers.exception.KubeClientException: Failed to delete pods for mutex key: 629af1e6-cc76-468b-a8cb-c17a17c778e8. stackTrace: [Ljava.lang.StackTraceElement;@2ee53b91 i see sync not happening and pod is failing to launch, any idea?

The knowledge sources do not explicitly mention this exact error (

Failed to delete pods for mutex key

), but based on the related issues available, here's what's likely happening and what you can try: What's Happening The

KubeClientException: Failed to delete pods for mutex key

error occurs during the MUTEX stage of the workload launcher pipeline. Airbyte uses a mutex mechanism to ensure only one replication pod runs per connection — before launching a new pod, it tries to delete any existing pods for that connection. In your case, this deletion is failing, which prevents the new sync pod from being launched. [KubeClientException issue] Troubleshooting Steps 1. *Check Kubernetes events for failures:*kubectl get events --namespace airbyte-namespace --field-selector reason=FailedCreate 2. Check if there are stuck/orphaned pods for that connection UUID (

629af1e6-cc76-468b-a8cb-c17a17c778e8

):kubectl get pods --namespace airbyte-namespace | grep 629af1e6If any pods are stuck in a bad state (e.g.,

Terminating

,

Unknown

), try force-deleting them:kubectl delete pod pod-name --namespace airbyte-namespace --force --grace-period=0 3. Check workload-launcher logs for more context:kubectl logs -l app.kubernetes.io/name=workload-launcher --namespace airbyte-namespace --tail=200 4. *Check node resource availability:*kubectl describe nodes | grep -A 5 "Allocated resources" 5. Check for RBAC/permission issues — the workload-launcher service account may lack permissions to delete pods in the namespace. Common Root Causes • Stuck pods in

Terminating

state that Kubernetes cannot clean up (often due to finalizers or node issues) • RBAC permissions preventing the launcher from deleting pods • Node pressure or storage issues causing Kubernetes API calls to fail [false positive liveness probes issue] • Cluster autoscaler evicting pods mid-operation [GKE autoscaler issue] Could you share your Airbyte/Helm chart version, Kubernetes provider, and any additional logs from the workload-launcher? That would help narrow down the root cause further. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.