what can be the reason of a sync to suddenly fail without any clear reason ?(k8s, 0.40.32)

2023-05-24 19:35:38 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$6):231 - Running sync worker cancellation...
2023-05-24 19:35:38 WARN i.a.w.s.LauncherWorker(killRunningPodsForConnection):253 - There are currently running pods for the connection: [orchestrator-repl-job-25900-attempt-0]. Killing these pods to enforce one execution at a time.
2023-05-24 19:35:38 INFO i.a.w.s.LauncherWorker(killRunningPodsForConnection):256 - Attempting to delete pods: [orchestrator-repl-job-25900-attempt-0]
2023-05-24 19:35:38 INFO i.a.w.s.LauncherWorker(killRunningPodsForConnection):261 - Waiting for deletion...
2023-05-24 19:35:39 WARN i.a.w.s.LauncherWorker(killRunningPodsForConnection):253 - There are currently running pods for the connection: [orchestrator-repl-job-25900-attempt-0]. Killing these pods to enforce one execution at a time.
2023-05-24 19:35:39 INFO i.a.w.s.LauncherWorker(killRunningPodsForConnection):256 - Attempting to delete pods: [orchestrator-repl-job-25900-attempt-0]
2023-05-24 19:35:39 INFO i.a.w.s.LauncherWorker(killRunningPodsForConnection):261 - Waiting for deletion...
2023-05-24 19:35:39 INFO i.a.w.p.AsyncOrchestratorPodProcess(computeExitValue):129 - State Store missing status. Orchestrator pod orchestrator-repl-job-25900-attempt-0 non-existent. Assume failure.
2023-05-24 19:35:39 INFO i.a.w.s.LauncherWorker(lambda$run$3):227 - Destroying process due to cancellation.
2023-05-24 19:35:39 WARN i.a.w.p.AsyncOrchestratorPodProcess(destroy):186 - Wasn't able to delete pod orchestrator-repl-job-25900-attempt-0 from namespace default
2023-05-24 19:35:39 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):316 - Stopping temporal heartbeating...

A message from kapa.ai