My setup: • Flink 2.2 • Cluster in session mode • Single JAR uploaded • Batch job started (repeatedly) using the same JAR artifact (triggered via REST API) After batch job has been executed enough number of times, task manager will eventually run out of heap memory. Heap dumps, point to

java.util.zip.ZipFile$Source

resources being held (one for each job executed). Based on everything I've been able to determine, I believe it's the

flink-pekko.actor.default-dispatcher

thread which is keeping these resources from being GC'ed!? So the question, has anyone experienced this problem? Anything which can be done to allow

ZipFile

to free up the artifact? Path in question

Class Name                                                                                                                                                                                 | Shallow Heap | Retained Heap
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
java.util.zip.ZipFile$Source @ 0x7584cef60                                                                                                                                                 |           80 |     2,970,368
|- zsrc java.util.zip.ZipFile$CleanableResource @ 0x75872d520                                                                                                                              |           32 |           360
|  |- res sun.net.www.protocol.jar.URLJarFile @ 0x75872d3f8                                                                                                                                |           80 |           352
|  |  |- value java.util.HashMap$Node @ 0x758408560                                                                                                                                        |           32 |           264
|  |  |  '- [10] java.util.HashMap$Node[16] @ 0x755c69638                                                                                                                                  |           80 |         2,048
|  |  |     '- table java.util.HashMap @ 0x755c69608                                                                                                                                       |           48 |         2,096
|  |  |        '- fileCache class sun.net.www.protocol.jar.JarFileFactory @ 0x755c628a8 System Class                                                                                       |           24 |         3,720
|  |  |- key java.util.HashMap$Node @ 0x758408668                                                                                                                                          |           32 |           144
|  |  |  '- [9] java.util.HashMap$Node[16] @ 0x755c69498                                                                                                                                   |           80 |         1,520
|  |  |     '- table java.util.HashMap @ 0x755c69468                                                                                                                                       |           48 |         1,568
|  |  |        '- urlCache class sun.net.www.protocol.jar.JarFileFactory @ 0x755c628a8 System Class                                                                                        |           24 |         3,720
|  |  '- Total: 2 entries                                                                                                                                                                  |              |              
|  |- action jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x75872d3c8                                                                                                                |           48 |            48
|  |  |- prev java.io.FileCleanable @ 0x7584cedf0                                                                                                                                          |           56 |            56
|  |  |  '- prev jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x758644a88                                                                                                            |           48 |            80
|  |  |     '- prev, discovered jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x758644ab8                                                                                             |           48 |            80
|  |  |        '- discovered jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x7586ffc68                                                                                                |           48 |            80
|  |  |           '- prev, discovered jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x7586ffc98                                                                                       |           48 |            80
|  |  |              '- discovered jdk.internal.ref.Cleaner @ 0x758534158                                                                                                                  |           40 |            72
|  |  |                 '- first class jdk.internal.ref.Cleaner @ 0x7ffd0d2a0 System Class                                                                                                 |            8 |             8
|  |  |- next jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x75872d340                                                                                                               |           48 |            48
|  |  |  '- next jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x75872d310                                                                                                            |           48 |            80
|  |  |     '- next jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x75872d2e0                                                                                                         |           48 |            80
|  |  |        '- next jdk.internal.ref.CleanerImpl$PhantomCleanableRef @ 0x75853a230                                                                                                      |           48 |            72
|  |  |           |- cleanable sun.nio.fs.NativeBuffer @ 0x7584c8708                                                                                                                       |           32 |           184
|  |  |           |  '- [0] sun.nio.fs.NativeBuffer[3] @ 0x7584c86e8                                                                                                                       |           32 |            32
|  |  |           |     '- value java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x7584c86c8                                                                                                  |           32 |            64
|  |  |           |        '- [12] java.lang.ThreadLocal$ThreadLocalMap$Entry[16] @ 0x757c63628                                                                                            |           80 |        16,656
|  |  |           |           '- table java.lang.ThreadLocal$ThreadLocalMap @ 0x757c63610                                                                                                  |           24 |        16,680
|  |  |           |              '- threadLocals org.apache.pekko.dispatch.MonitorableThreadFactory$PekkoForkJoinWorkerThread @ 0x7556325d0  flink-pekko.actor.default-dispatcher-18 Thread|          120 |        16,896
|  |  |           |- next java.io.FileCleanable @ 0x75853a1f8                                                                                                                              |           56 |            56
|  |  |           '- Total: 2 entries                                                                                                                                                      |              |              
|  |  '- Total: 2 entries                                                                                                                                                                  |              |              
|  '- Total: 2 entries                                                                                                                                                                     |              |              
|- value java.util.HashMap$Node @ 0x75875ac30                                                                                                                                              |           32 |            32
|  '- next java.util.HashMap$Node @ 0x755d0f700                                                                                                                                            |           32 |            64
|     '- next java.util.HashMap$Node @ 0x755d0f6e0                                                                                                                                         |           32 |            96
|        '- [3] java.util.HashMap$Node[64] @ 0x755d0f2c8                                                                                                                                   |          272 |         1,360
|           '- table java.util.HashMap @ 0x755d060d0                                                                                                                                       |           48 |         1,408
|              '- files class java.util.zip.ZipFile$Source @ 0x7ffd2d6c0 System Class                                                                                                      |           40 |         3,048
'- Total: 2 entries                                                                                                                                                                        |              |              
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------