Hi! I am using Flink version 1.17 and batch mode t...
# troubleshooting
l
Hi! I am using Flink version 1.17 and batch mode to write ORC files through the file system connector. How can I control the size of the written ORC files? I found that only one ORC file is generated, and as the amount of data on the source side increases, the written file becomes larger, resulting in a Java heap memory overflow exception。
p
1- Choose the right number of parallelism. env.setParallelism(n) 2- You can also explore bucket assigned and see if that helps to reduce file size
l
Thank you. I'll have a try.