Hello Team, I’m encountering issues with autoscali...
# troubleshooting
m
Hello Team, I’m encountering issues with autoscaling, particularly with the following: 1. The TaskManager is not scaling down, even when the load is minimal. It is taking over an hour to scale down. 2. Despite having free slots in the TaskManager, it’s not releasing or scaling down the TaskManager resources. Below are the configurations I’m currently using for autoscaling.
Copy code
flinkConfiguration: 
    taskmanager.numberOfTaskSlots: "5"
    kubernetes.operator.job.autoscaler.enabled: "true"
    kubernetes.operator.job.autoscaler.scaling.enabled: "true"
    kubernetes.operator.job.autoscaler.stabilization.interval: 2m
    kubernetes.operator.job.autoscaler.metrics.window: 1m
    kubernetes.operator.job.autoscaler.target.utilization: "0.6"
    kubernetes.operator.job.autoscaler.target.utilization.boundary: "0.1"
    kubernetes.operator.job.autoscaler.restart.time: 2m
    kubernetes.operator.job.autoscaler.catch-up.duration: 3m
    pipeline.max-parallelism: "50"
    jobmanager.scheduler: adaptive
    kubernetes.operator.resource.metrics.enabled: "true"
    kubernetes.operator.resource.lifecycle.metrics.enabled: "true"
    scaling.effectiveness.detection.enabled: "true"
reference: https://github.com/apache/flink-kubernetes-operator/blob/main/flink-autoscaler/src[…]/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java It would be helpful if someone from the team could take a look into why the TaskManager isn’t being released even when there are free slots.
r
https://github.com/apache/flink/pull/25218 I guess this PR and FLINK-33977 is fixing the issue that you mentioned.
In brief, the tm can be released when all slots are free. It cannot be released when any slot is using.
m
Thanks for the reply, @Rui Fan! It’s great to hear this is already in the plan. Could you let us know when it’s expected to go live, or in which version? Also, my first question is open. Could you take a look at that as well? Thanks
r
The solution is not determined for now, so I’m not sure when it will be merged. It’s better to subscribe the PR. Also, welcome to feedback your issue to that PR and discuss together.
👍 1
m
Sure thanks.