hey All, Sweeper Pod is not deleting the Pods and ...
# replication-troubleshooting
m
hey All, Sweeper Pod is not deleting the Pods and also when I check the logs, it’s not even generating any logs
m
This is a new Airbyte instance or a long-running. What version of Helm or Kustomize are you using?
m
Hi @Marcos Marx (Airbyte) This is a new Airbyte instance, and the helm version is 3.10.3 and the chart version is 0.43.18
Also pasting the config script for sweeper pod here
#!/bin/bash get_worker_pods () { kubectl -n ${KUBE_NAMESPACE} -L airbyte -l airbyte=worker-pod \ --field-selector status.phase!=Running get pods \ -o=jsonpath='{range .items[*]} {.metadata.name} {.status.phase} {.status.conditions[0].lastTransitionTime} {.status.startTime}{"\n"}{end}' } delete_worker_pod() { printf "From status '%s' since '%s', " $2 $3 echo "$1" | grep -v "STATUS" | awk '{print $1}' | xargs --no-run-if-empty kubectl -n ${KUBE_NAMESPACE} delete pod } while : do # Shorter time window for completed pods SUCCESS_DATE_STR=
date -d 'now - 120 minutes' --utc -Ins
SUCCESS_DATE=
date -d $SUCCESS_DATE_STR +%s
# Longer time window for pods in error (to debug) NON_SUCCESS_DATE_STR=
date -d 'now - 1440 minutes' --utc -Ins
NON_SUCCESS_DATE=
date -d $NON_SUCCESS_DATE_STR +%s
( IFS=$'\n' for POD in `get_worker_pods`; do IFS=' ' POD_NAME=
echo $POD | cut -d " " -f 1
POD_STATUS=
echo $POD | cut -d " " -f 2
POD_DATE_STR=
echo $POD | cut -d " " -f 3
POD_START_DATE_STR=
echo $POD | cut -d " " -f 4
POD_DATE=
date -d ${POD_DATE_STR:-$POD_START_DATE_STR} '+%s'
if [ "$POD_STATUS" = "Succeeded" ]; then if [ "$POD_DATE" -lt "$SUCCESS_DATE" ]; then delete_worker_pod "$POD_NAME" "$POD_STATUS" "$POD_DATE_STR" fi else if [ "$POD_DATE" -lt "$NON_SUCCESS_DATE" ]; then delete_worker_pod "$POD_NAME" "$POD_STATUS" "$POD_DATE_STR" fi fi done ) sleep 60 done
Also when I run Helm search repo airbyte
NAME CHART VERSION APP VERSION DESCRIPTION airbyte/airbyte 0.43.18 0.40.27 Helm chart to deploy airbyte airbyte/airbyte-bootloader 0.43.18 0.40.27 Helm chart to deploy airbyte-bootloader airbyte/airbyte-cron 0.40.37 0.40.17 Helm chart to deploy airbyte-cron airbyte/connector-builder-server 0.43.18 0.40.27 Helm chart to deploy airbyte-connector-builder-... airbyte/cron 0.43.18 0.40.27 Helm chart to deploy airbyte-cron airbyte/metrics 0.43.18 0.40.3 Helm chart to deploy airbyte-metrics airbyte/pod-sweeper 0.43.18 0.39.37-alpha Helm chart to deploy airbyte-pod-sweeper airbyte/server 0.43.18 0.40.27 Helm chart to deploy airbyte-server airbyte/temporal 0.43.18 0.40.27 Helm chart to deploy airbyte-temporal airbyte/webapp 0.43.18 0.40.27 Helm chart to deploy airbyte-webapp airbyte/worker 0.43.18 0.40.27 Helm chart to deploy airbyte-worker
I see two cron one with 0.40.37 version and another one with 0.43.18 version
u
Did you try to restart de pod-sweeper?
m
Yes I have tried, still it’s not deleting pods
@Marcos Marx (Airbyte) Any recommendation what should I do ?
m
I found this issue https://github.com/airbytehq/airbyte/issues/18588 can you confirme you’re not using arm64?
m
@Marcos Marx (Airbyte) This is the screenshot of my node
OS (Architecture) linux (amd64)
What Should I do now?