Hi All, I am using flink 1.16.0 version and trying...
# troubleshooting
f
Hi All, I am using flink 1.16.0 version and trying to set managed memory fraction here is snippet from my taskmanager flink-conf.yaml
Copy code
state.backend: rocksdb
state.backend.incremental: true
state.backend.rocksdb.metrics.enable: true
state.backend.rocksdb.write-buffer-size:128mb
state.backend.rocksdb.max-write-buffer-number:10
state.backend.rocksdb.checkpoint.transfer.thread.num: 4
taskmanager.memory.network.fraction:0.25
taskmanager.memory.managed.fraction:0.5
Since physical memory for this manager is 8GB shouldn't i get 4GB to managed memory on Flink UI or am i missing something ??
j
You're allocating half of the Flink's total memory to managed memory, not half of your machine's physical memory.
f
Thanks John..! When i specify following
taskmanager.memory.process.size: 8g
taskmanager.memory.managed.fraction: 0.75
I see the following logs.. as per doc fraction is of
Total Flink Memory
but that also includes managed memory
Copy code
2023-06-22 18:27:30 INFO  [] - Final TaskExecutor Memory configuration:
2023-06-22 18:27:30 INFO  [] -   Total Process Memory:          8.000gb (8589934592 bytes)
2023-06-22 18:27:30 INFO  [] -     Total Flink Memory:          6.950gb (7462505664 bytes)
2023-06-22 18:27:30 INFO  [] -       Total JVM Heap Memory:     1.265gb (1358283484 bytes)
2023-06-22 18:27:30 INFO  [] -         Framework:               128.000mb (134217728 bytes)
2023-06-22 18:27:30 INFO  [] -         Task:                    1.140gb (1224065756 bytes)
2023-06-22 18:27:30 INFO  [] -       Total Off-heap Memory:     5.685gb (6104222180 bytes)
2023-06-22 18:27:30 INFO  [] -         Managed:                 4.865gb (5223753875 bytes)
2023-06-22 18:27:30 INFO  [] -         Total JVM Direct Memory: 839.680mb (880468305 bytes)
2023-06-22 18:27:30 INFO  [] -           Framework:             128.000mb (134217728 bytes)
2023-06-22 18:27:30 INFO  [] -           Task:                  0 bytes
2023-06-22 18:27:30 INFO  [] -           Network:               711.680mb (746250577 bytes)
2023-06-22 18:27:30 INFO  [] -     JVM Metaspace:               256.000mb (268435456 bytes)
2023-06-22 18:27:30 INFO  [] -     JVM Overhead:                819.200mb (858993472 bytes)