Apache Flink

Does anyone know how to configure flink task manager memory for EMR EC2 instances?
I am using `c6i.xlarge` which has `5632` mb memory as per <https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hadoop-task-config.html#emr-hadoop-task-config-c7g|docs>
And I have set memory as
```taskmanager.memory.process.size = 4g
taskmanager.numberOfTaskSlots = 2```
However it was not able to spawn a task manager in the instance...when I change to `c5.xlarge` which has `6144` mb it is able to spawn the TM
So now my question is i anyone has any tips on how you calculate task manager memory based on instance type and available RAM?

Hi, maybe this can help: <https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/deployment/memory/mem_setup_tm/>

That only documents how we set memory and not how much we can set right?
For example I am looking for something like we have for Spark:
<https://aws.github.io/aws-emr-best-practices/applications/spark/best_practices/#bp-516-tune-driverexecutor-memory-cores-and-sparksqlshufflepartitions-to-fully-utilize-cluster-resources|https://aws.github.io/aws-emr-best-practices/applications/spark/best_practices/#bp-516-tune-driverexecutor-memor[…]o-fully-utilize-cluster-resources>

Ok I got it...its because job manager is using 1600mb and task manager needs 4096mb which is 5696mb in total. c6i.xlarge only has 5632...which is why its not able to spawn the TM.

cool, glad you figured it out :slightly_smiling_face: