Hussain Abbas
05/05/2023, 4:18 PMreactive scheduler and standalone mode. We are using HPA to scale up and down the task managers based on metrics, while flink takes cares of vertical scaling.
Our pipeline consists of Kinesis as source and Dynamo as Sink. There are few issues we are seeing right now.
1- We are getting failing checkpoints with unalligned checkpoints, which causes taskmanager restart
2- Task manager restarts on high load not sure what is causing though as we cannot see any exceptions
We have a very high load environments 10-12k events/s. Do you think there is any room for improvement? Are we handling it correctly?Hussain Abbas
05/10/2023, 8:58 AMHussain Abbas
05/11/2023, 10:04 AMSamrat Deb
05/11/2023, 10:43 AMHussain Abbas
05/12/2023, 2:39 PM