Apache Flink

Screenshot 2023-03-23 at 11.56.27 AM.png

Hello! Trying to figure out what parameters to tweak next... I have a Flink application that reads Protobuf messages from Kafka topic then these events are persisted to S3. We are running Flink via the Lyft Flink Kubernetes Operator. I have <https://github.com/lyft/flinkk8soperator/blob/435640258b72d9c9efdbadc3579411224f510a72/examples/beam-python/flink-operator-custom-resource.yaml#L32|parallelism set to 10> compared to 1 for our staging env. Of course, our staging env sees *far* fewer events than prod.

I'm getting `Checkpoint expired before completing`

considering increasing CPU for the deployment, or using unaligned checkpoints but not sure what makes most sense here