Hey Team 👋 Looking for some guidance from the flink community on managing flink deployments on Kubernetes. We recently migrated our flink workloads from a Hadoop VM based cluster to a Kubernetes Operator-managed deployment. We are currently observing high memory consumption in the Flink containers. Our containers also run additional observability agents, which require their own memory overhead. The challenge is that when we configure the total process memory for the Jobmanager and Taskmanager, the resulting container sizing doesn’t leave enough headroom for these additional agents. Has anyone faced a similar situation or have recommendations on how to configure flink memory/process memory. Any guidance or best practices would be greatly appreciated. Thanks!

Hello there, umar! Flink has a the configs

(jobmanager|taskmanager).memory.jvm-overhead.fraction

I think this may help you with this problem. Every time you increase the memory, the default behaviour is to use everything for the "main" assets. If this agent could be a sidecar, this would be the best option to be sure they're separated in regards of resources.

Thanks @Pedro Mázala, Sidecard is not an option we have . We did tried jvm over head fraction . Still the memory consumption by the Flink workload is keeping increasing we are trying to see what would be best approach to give right sizing for flink workloads