Hey folks :wave: Using Apache Flink Operator versi...
# troubleshooting
a
Hey folks πŸ‘‹ Using Apache Flink Operator version 1.5.0. For some reason after submitting my job and having it
DEPLOYED
and
STABLE
there is a long delay (10 to 20 minutes) before the `TaskManager`(s) get created 🧡
Here are the controller logs immediately following that state transition
Copy code
2023-10-12 21:36:17,675 o.a.f.k.o.l.AuditUtils         [INFO ][default/recipe] >>> Status | Info    | DEPLOYED        | The resource is deployed/submitted to Kubernetes, but it's not yet considered to be stable and might be rolled back in the future 
2023-10-12 21:36:17,682 o.a.f.k.o.r.d.AbstractFlinkResourceReconciler [INFO ][default/recipe] Resource fully reconciled, nothing to do...
2023-10-12 21:36:17,682 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] End of reconciliation
2023-10-12 21:36:27,702 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] Starting reconciliation
2023-10-12 21:36:27,707 o.a.f.k.o.s.FlinkResourceContextFactory [INFO ][default/recipe] Getting service for recipe
2023-10-12 21:36:27,708 o.a.f.k.o.o.d.SessionObserver  [INFO ][default/recipe] Observing JobManager deployment. Previous status: DEPLOYED_NOT_READY
2023-10-12 21:36:27,708 o.a.f.k.o.o.d.SessionObserver  [INFO ][default/recipe] JobManager deployment is ready
2023-10-12 21:36:28,207 o.a.f.k.o.l.AuditUtils         [INFO ][default/recipe] >>> Status | Info    | STABLE          | The resource deployment is considered to be stable and won't be rolled back 
2023-10-12 21:36:28,226 o.a.f.k.o.r.d.AbstractFlinkResourceReconciler [INFO ][default/recipe] Resource fully reconciled, nothing to do...
2023-10-12 21:36:28,227 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] End of reconciliation
^[[B^[[B^[[B^[[B2023-10-12 21:37:28,241 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] Starting reconciliation
2023-10-12 21:37:28,250 o.a.f.k.o.s.FlinkResourceContextFactory [INFO ][default/recipe] Getting service for recipe
2023-10-12 21:37:28,348 o.a.f.k.o.r.d.AbstractFlinkResourceReconciler [INFO ][default/recipe] Resource fully reconciled, nothing to do...
2023-10-12 21:37:28,348 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] End of reconciliation
2023-10-12 21:38:28,363 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] Starting reconciliation
2023-10-12 21:38:28,369 o.a.f.k.o.s.FlinkResourceContextFactory [INFO ][default/recipe] Getting service for recipe
2023-10-12 21:38:28,478 o.a.f.k.o.r.d.AbstractFlinkResourceReconciler [INFO ][default/recipe] Resource fully reconciled, nothing to do...
2023-10-12 21:38:28,479 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] End of reconciliation
2023-10-12 21:39:28,489 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] Starting reconciliation
2023-10-12 21:39:28,494 o.a.f.k.o.s.FlinkResourceContextFactory [INFO ][default/recipe] Getting service for recipe
2023-10-12 21:39:28,583 o.a.f.k.o.r.d.AbstractFlinkResourceReconciler [INFO ][default/recipe] Resource fully reconciled, nothing to do...
2023-10-12 21:39:28,584 o.a.f.k.o.c.FlinkDeploymentController [INFO ][default/recipe] End of reconciliation
Since there is nothing to reconcile according to the logs what could be happening here that takes so long for the job's tasks to kick off?
c
Can you check the FlinkDeployment events?
a
@Carlos Santos: I did, very little information about anything there