https://linen.dev logo
Join Slack
Powered by
Why am I getting these issues on my VM ? ```abctl...
# replication-ask-ai
j
Why am I getting these issues on my VM ?
Copy code
abctl local install --insecure-cookies --low-resource-mode
  INFO    Using Kubernetes provider:
            Provider: kind
            Kubeconfig: /root/.airbyte/abctl/abctl.kubeconfig
            Context: kind-airbyte-abctl
 SUCCESS  Found Docker installation: version 26.1.4                                                                                                                                 
  INFO    No existing cluster found, cluster 'airbyte-abctl' will be created                                                                                                        
 SUCCESS  Port 8000 appears to be available                                                                                                                                         
 SUCCESS  Cluster 'airbyte-abctl' created                                                                                                                                           
  INFO    Patching image airbyte/db:1.7.0-17                                                                                                                                        
  INFO    Pulling image airbyte/async-profiler:2.0.1                                                                                                                                
  INFO    Pulling image airbyte/bootloader:2.0.1                                                                                                                                    
  INFO    Pulling image airbyte/connector-sidecar:2.0.1                                                                                                                             
  INFO    Pulling image airbyte/container-orchestrator:2.0.1                                                                                                                        
  INFO    Pulling image airbyte/cron:2.0.1                                                                                                                                          
  INFO    Pulling image airbyte/db:1.7.0-17                                                                                                                                         
  INFO    Pulling image airbyte/server:2.0.1                                                                                                                                        
  INFO    Pulling image airbyte/utils:2.0.1                                                                                                                                         
  INFO    Pulling image airbyte/worker:2.0.1                                                                                                                                        
  INFO    Pulling image airbyte/workload-api-server:2.0.1                                                                                                                           
  INFO    Pulling image airbyte/workload-init-container:2.0.1                                                                                                                       
  INFO    Pulling image airbyte/workload-launcher:2.0.1                                                                                                                             
  INFO    Pulling image temporalio/auto-setup:1.27.2                                                                                                                                
  INFO    Namespace 'airbyte-abctl' created                                                                                                                                         
  INFO    Persistent volume 'airbyte-local-pv' created                                                                                                                              
  INFO    Persistent volume claim 'airbyte-storage-pvc' created                                                                                                                     
  INFO    Persistent volume 'airbyte-volume-db' created                                                                                                                             
  INFO    Persistent volume claim 'airbyte-volume-db-airbyte-db-0' created                                                                                                          
  INFO    Starting Helm Chart installation of 'airbyte/airbyte' (version: 2.0.19)                                                                                                   
 WARNING  Encountered an issue deploying Airbyte:                                                                                                                                   
            Pod: airbyte-abctl-worker-7c698f674c-9s55p.18a3d2fb46b88bec
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.8:9000/>": dial tcp 10.244.0.8:9000: connect: connection refused
            Count: 6
 WARNING  Encountered an issue deploying Airbyte:                                                                                                                                   
            Pod: airbyte-abctl-workload-api-server-5ccbbb5689-vgw99.18a3d2fb46b6f459
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.9:8085/health/liveness>": dial tcp 10.244.0.9:8085: connect: connection refused
            Count: 6
 WARNING  Encountered an issue deploying Airbyte:                                                                                                                                   
            Pod: airbyte-abctl-server-7fd587f787-8gjz7.18a3d2fb46b303e0
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.11:8001/api/v1/health>": dial tcp 10.244.0.11:8001: connect: connection refused
            Count: 6
k
The warnings you're seeing (
Readiness probe failed: ... connect: connection refused
) are a known issue during Airbyte installation. These warnings indicate that some pods (worker, workload-api-server, server) are not yet ready when the readiness probes run, but this doesn't necessarily mean the installation has failed. A few things to consider: 1. This may be a resource issue You're running in 
--low-resource-mode
, which is intended for environments with fewer than 4 CPUs. However, even in low-resource mode, pods can take time to start up and may temporarily fail readiness probes. The Airbyte documentation notes that if you see readiness probe failures, installation can still complete — you may just need to wait. [OSS Quickstart] 2. These are `WARNING`s, not fatal errors The output shows 
WARNING Encountered an issue deploying Airbyte
— not a hard failure. The documentation explicitly states: > If you see the warning 
Encountered an issue deploying Airbyte
with the message 
Readiness probe failed
, allow installation to continue. You may need to allocate more resources for Airbyte, but installation can complete anyway. [OSS Quickstart] 3. Docker version may be a factor You're using Docker version 26.1.4. There is a known issue where older Docker versions can cause cluster initialization problems. One GitHub issue noted that upgrading to Docker 28.x resolved a similar cluster health check failure. [GitHub Issue] Recommended steps: • Wait for the installation to fully complete — it can take up to 30 minutes. • If it ultimately fails, try uninstalling and reinstalling:abctl local uninstall abctl local install --insecure-cookies --low-resource-mode • Consider upgrading Docker to version 28.x or later. • Ensure your VM has sufficient resources allocated (at minimum 4 CPUs and adequate RAM are recommended for a standard install). 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
j
Now I have this @kapa.ai
Copy code
/   |  (_)____/ /_  __  __/ /____
            / /| | / / ___/ __ \/ / / / __/ _ \
           / ___ |/ / /  / /_/ / /_/ / /_/  __/
          /_/  |_/_/_/  /_.___/\__, /\__/\___/
                              /____/
                  : airbyte-server :
          --------------------------------------------------
           Will soon be available at <http://localhost:8000/>
          --------------------------------------------------
          
          2026-04-06 17:30:22,512 [main]        INFO    i.m.c.e.DefaultEnvironment(<init>):170 - Established active environments: [k8s, control-plane, edition-community]
          2026-04-06 17:30:32,898 [main]        WARN    c.z.h.HikariConfig(validateNumerics):1135 - HikariPool-1 - idleTimeout is close to or more than maxLifetime, disabling it.
          2026-04-06 17:30:32,979 [main]        INFO    c.z.h.HikariDataSource(<init>):79 - HikariPool-1 - Starting...
          2026-04-06 17:30:33,125 [main]        INFO    c.z.h.HikariDataSource(<init>):81 - HikariPool-1 - Start completed.
          2026-04-06 17:30:34,800 [main]        INFO    i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):163 - Waiting for temporal server...
          2026-04-06 17:30:34,802 [main]        WARN    i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):174 - Waiting for namespace default to be initialized in temporal...
 WARNING  Encountered an issue deploying Airbyte:                                                                                                                                   
            Pod: airbyte-abctl-worker-7c698f674c-9s55p.18a3d34dba876e73
            Reason: BackOff
            Message: Back-off restarting failed container airbyte-worker-container in pod airbyte-abctl-worker-7c698f674c-9s55p_airbyte-abctl(4f32483b-c4ba-4726-9d73-81526fc6b6b4)
            Count: 106
            Logs: Unsetting empty environment variable 'CHECK_JOB_MAIN_CONTAINER_MEMORY_LIMIT'
          Unsetting empty environment variable 'ACTIVITY_MAX_DELAY_BETWEEN_ATTEMPTS_SECONDS'
          Unsetting empty environment variable 'CHECK_JOB_MAIN_CONTAINER_CPU_REQUEST'
          Unsetting empty environment variable 'DISCOVER_JOB_MAIN_CONTAINER_MEMORY_REQUEST'
          Unsetting empty environment variable 'AIRBYTE_CLUSTER_NAME'
          Unsetting empty environment variable 'OTEL_COLLECTOR_ENDPOINT'
          Unsetting empty environment variable 'JOB_KUBE_TOLERATIONS'
          Unsetting empty environment variable 'SIDECAR_MAIN_CONTAINER_MEMORY_REQUEST'
          Unsetting empty environment variable 'JOB_MAIN_CONTAINER_CPU_REQUEST'
          Unsetting empty environment variable 'REPLICATION_ORCHESTRATOR_CPU_REQUEST'
          Unsetting empty environment variable 'JOB_KUBE_MAIN_CONTAINER_IMAGE_PULL_SECRET'
          Unsetting empty environment variable 'REPLICATION_ORCHESTRATOR_MEMORY_LIMIT'
          Unsetting empty environment variable 'JOB_MAIN_CONTAINER_MEMORY_REQUEST'
          Unsetting empty environment variable 'SIDECAR_MAIN_CONTAINER_CPU_REQUEST'
          Unsetting empty environment variable 'ACTIVITY_MAX_ATTEMPT'
          Unsetting empty environment variable 'JOB_KUBE_ANNOTATIONS'
          Unsetting empty environment variable 'JOB_MAIN_CONTAINER_CPU_LIMIT'
          Unsetting empty environment variable 'DISCOVER_REFRESH_WINDOW_MINUTES'
          Unsetting empty environment variable 'DISCOVER_JOB_MAIN_CONTAINER_CPU_REQUEST'
          Unsetting empty environment variable 'JOB_KUBE_LABELS'
          Unsetting empty environment variable 'REPLICATION_ORCHESTRATOR_MEMORY_REQUEST'
          Unsetting empty environment variable 'VERSION'
          Unsetting empty environment variable 'WORKFLOW_FAILURE_RESTART_DELAY_SECONDS'
          Unsetting empty environment variable 'DISCOVER_JOB_MAIN_CONTAINER_MEMORY_LIMIT'
          Unsetting empty environment variable 'CHECK_JOB_MAIN_CONTAINER_CPU_LIMIT'
          Unsetting empty environment variable 'SIDECAR_MAIN_CONTAINER_CPU_LIMIT'
          Unsetting empty environment variable 'REPLICATION_ORCHESTRATOR_CPU_LIMIT'
          Unsetting empty environment variable 'SECRET_PERSISTENCE'
          Unsetting empty environment variable 'DEPLOYMENT_ENV'
          Unsetting empty environment variable 'SIDECAR_MAIN_CONTAINER_MEMORY_LIMIT'
          Unsetting empty environment variable 'DISCOVER_JOB_MAIN_CONTAINER_CPU_LIMIT'
          Unsetting empty environment variable 'JOB_MAIN_CONTAINER_MEMORY_LIMIT'
          Unsetting empty environment variable 'CHECK_JOB_MAIN_CONTAINER_MEMORY_REQUEST'
          Unsetting empty environment variable 'JOB_KUBE_NODE_SELECTORS'
          Unsetting empty environment variable 'ACTIVITY_INITIAL_DELAY_BETWEEN_ATTEMPTS_SECONDS'
          
              ___    _      __          __
             /   |  (_)____/ /_  __  __/ /____
            / /| | / / ___/ __ \/ / / / __/ _ \
           / ___ |/ / /  / /_/ / /_/ / /_/  __/
          /_/  |_/_/_/  /_.___/\__, /\__/\___/
                              /____/
                  : airbyte-workers :
          
          2026-04-06 17:30:03,150 [main]        INFO    i.m.c.e.DefaultEnvironment(<init>):170 - Established active environments: [k8s, control-plane, edition-community]
          2026-04-06 17:30:10,863 [main]        INFO    c.z.h.HikariDataSource(<init>):79 - config-pool - Starting...
          2026-04-06 17:30:11,110 [main]        INFO    c.z.h.HikariDataSource(<init>):81 - config-pool - Start completed.
          2026-04-06 17:30:12,566 [main]        INFO    i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'INFO' for logger: 'io.netty'
          2026-04-06 17:30:12,585 [main]        INFO    i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'ERROR' for logger: 'com.zaxxer.hikari'
          2026-04-06 17:30:12,589 [main]        INFO    i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'INFO' for logger: 'io.grpc'
          2026-04-06 17:30:12,593 [main]        INFO    i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'INFO' for logger: 'io.temporal'
          2026-04-06 17:30:12,597 [main]        INFO    i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'ERROR' for logger: 'com.zaxxer.hikari.pool'
          2026-04-06 17:30:12,598 [main]        INFO    i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'INFO' for logger: 'io.fabric8.kubernetes.client'
 WARNING  Encountered an issue deploying Airbyte:                                                                                                                                   
            Pod: airbyte-abctl-workload-launcher-75c9c79fc4-5ggjh.18a3d34bb976409e
            Reason: BackOff
            Message: Back-off restarting failed container airbyte-workload-launcher-container in pod airbyte-abctl-workload-launcher-75c9c79fc4-5ggjh_airbyte-abctl(f89652b8-b141-4d5f-8d32-eb901a65e122)
            Count: 107
            Logs: Unsetting empty environment variable 'JOB_ISOLATED_KUBE_NODE_SELECTORS'
k
The logs you're sharing show two key issues: 1. The 
airbyte-worker
pod is crash-looping (
Back-off restarting failed container
) — it starts, logs a bunch of "Unsetting empty environment variable" messages, and then exits/restarts repeatedly. 2. The 
airbyte-workload-launcher
pod is also crash-looping for similar reasons. The worker logs show it connects to the database successfully but then appears to be waiting on Temporal (
Waiting for temporal server...
/ 
Waiting for namespace default to be initialized in temporal...
). This suggests the Temporal service may not be healthy, which is causing the worker to fail to initialize and restart in a loop. Unfortunately, the knowledge sources I have available don't contain enough specific information to definitively diagnose this exact crash-loop pattern (worker/workload-launcher failing while waiting for Temporal). However, based on the available sources, here are the recommended troubleshooting steps: 1. Try a clean reinstall This is the most commonly suggested fix for persistent deployment issues: [abctl install stuck; Deployment not ready]
Copy code
abctl local uninstall
abctl local install --insecure-cookies --low-resource-mode
2. Check available disk space and Docker resources Image import failures and pod crashes can be caused by insufficient disk space: [bootloader failed]
Copy code
df -h
docker system df
3. Check pod status and logs directly
Copy code
kubectl get pods -n airbyte-abctl --kubeconfig /root/.airbyte/abctl/abctl.kubeconfig
kubectl logs airbyte-abctl-temporal-0 -n airbyte-abctl --kubeconfig /root/.airbyte/abctl/abctl.kubeconfig
Checking the Temporal pod logs specifically may reveal why the worker cannot connect to it. 4. Check your VM's available memory One GitHub issue showed that a server with 15GB RAM but only ~4GB available (due to other running applications) caused similar deployment failures. [Deployment not ready] Make sure your VM has sufficient free memory for Airbyte to run. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
j
@kapa.ai Here are my kubectl logs :
Copy code
kubectl get pods -n airbyte-abctl --kubeconfig /root/.airbyte/abctl/abctl.kubeconfig
kubectl logs airbyte-abctl-temporal-0 -n airbyte-abctl --kubeconfig /root/.airbyte/abctl/abctl.kubeconfig
NAME                                                 READY   STATUS             RESTARTS       AGE
airbyte-abctl-bootloader                             0/1     Completed          0              43m
airbyte-abctl-cron-fcfbd4979-ch9df                   1/1     Running            0              41m
airbyte-abctl-server-7fd587f787-8gjz7                0/1     CrashLoopBackOff   15 (31s ago)   41m
airbyte-abctl-temporal-645f495958-rwl4q              1/1     Running            0              41m
airbyte-abctl-worker-7c698f674c-9s55p                0/1     CrashLoopBackOff   15 (42s ago)   41m
airbyte-abctl-workload-api-server-5ccbbb5689-vgw99   1/1     Running            9 (22m ago)    41m
airbyte-abctl-workload-launcher-75c9c79fc4-5ggjh     0/1     CrashLoopBackOff   15 (40s ago)   41m
airbyte-db-0                                         1/1     Running            0              42m
error: error from server (NotFound): pods "airbyte-abctl-temporal-0" not found in namespace "airbyte-abctl"
k
Looking at your pod status, the situation is clearer now: • Temporal is running as a Deployment pod (
airbyte-abctl-temporal-645f495958-rwl4q
), not a StatefulSet — that's why 
airbyte-abctl-temporal-0
was not found. • Three pods are in CrashLoopBackOff: 
server
, 
worker
, and 
workload-launcher
(15 restarts each). • Bootloader completed successfully (
0/1 Completed
) — so the DB migration ran fine. The fact that 
server
, 
worker
, and 
workload-launcher
are all crash-looping while Temporal itself is running suggests a resource constraint issue rather than a Temporal connectivity problem. Recommended steps: 1. Check the server pod logs — since the server is also crash-looping (which wasn't visible before), it may give more clues:
Copy code
kubectl logs airbyte-abctl-server-7fd587f787-8gjz7 -n airbyte-abctl --kubeconfig /root/.airbyte/abctl/abctl.kubeconfig --previous
2. Check available memory on your VM A shared server with insufficient free RAM is a known cause of this pattern. [Deployment not ready] Run:
Copy code
free -h
3. Try a clean reinstall This is the most commonly suggested resolution for persistent CrashLoopBackOff issues during installation: [Deployment not ready]
Copy code
abctl local uninstall
abctl local install --insecure-cookies --low-resource-mode
4. Check disk space
Copy code
df -h
docker system df
The knowledge sources don't contain enough specific information to pinpoint the exact root cause of the crash loops beyond these recommendations. Sharing the output of 
kubectl logs ... --previous
for the server pod would help narrow it down further. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
j
@kapa.ai Here are the logs :
Copy code
___    _      __          __
   /   |  (_)____/ /_  __  __/ /____
  / /| | / / ___/ __ \/ / / / __/ _ \
 / ___ |/ / /  / /_/ / /_/ / /_/  __/
/_/  |_/_/_/  /_.___/\__, /\__/\___/
                    /____/
        : airbyte-server :
--------------------------------------------------
 Will soon be available at <http://localhost:8000/>
--------------------------------------------------

2026-04-06 17:37:35,825 [main]  INFO    i.m.c.e.DefaultEnvironment(<init>):170 - Established active environments: [k8s, control-plane, edition-community]
2026-04-06 17:37:44,327 [main]  WARN    c.z.h.HikariConfig(validateNumerics):1135 - HikariPool-1 - idleTimeout is close to or more than maxLifetime, disabling it.
2026-04-06 17:37:44,372 [main]  INFO    c.z.h.HikariDataSource(<init>):79 - HikariPool-1 - Starting...
2026-04-06 17:37:44,435 [main]  INFO    c.z.h.HikariDataSource(<init>):81 - HikariPool-1 - Start completed.
2026-04-06 17:37:45,880 [main]  INFO    i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):163 - Waiting for temporal server...
2026-04-06 17:37:45,881 [main]  WARN    i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):174 - Waiting for namespace default to be initialized in temporal...
k
The server logs show it's stuck at:
Copy code
Waiting for temporal server...
Waiting for namespace default to be initialized in temporal...
This means the airbyte-server is unable to connect to or initialize the Temporal namespace, even though the Temporal pod (
airbyte-abctl-temporal-645f495958-rwl4q
) shows as 
Running
. This is a known pattern — other pods (server, worker, workload-launcher) repeatedly fail while waiting for Temporal to become fully ready. [Temporal connection failure] The knowledge sources don't contain a definitive root cause or fix specific to this exact scenario (Temporal pod running but namespace not initializing). However, based on what's available, here are the recommended steps: 1. Check Temporal pod logs directly (using the correct pod name):
Copy code
kubectl logs airbyte-abctl-temporal-645f495958-rwl4q -n airbyte-abctl --kubeconfig /root/.airbyte/abctl/abctl.kubeconfig
This may reveal why the 
default
namespace isn't being initialized inside Temporal. 2. Check your VM's free memory Resource pressure can prevent Temporal from fully initializing even if the pod shows as 
Running
. [Deployment not ready]
Copy code
free -h
3. Try a clean reinstall
Copy code
abctl local uninstall
abctl local install --insecure-cookies --low-resource-mode
This is the most commonly suggested resolution for persistent crash-loop issues during installation. [Deployment not ready] Please share the Temporal pod logs if the above steps don't resolve the issue — that would be the most useful next piece of information to diagnose further. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
7 Views
Open in Slack
PreviousNext
Powered by