Hi all! We updated the Flink version in our servic...
# troubleshootings
saqlain pasha
07/10/2023, 4:41 PMHi all! We updated the Flink version in our services from 1.13.6 to 1.17.0, but we are seeing considerable performance degradation. The same load that our services used to handle with 3 TMs with 2 CPU cores each (1.13.6 version) now requires either 3 CPU cores per TM or 5 TMs with 2 CPU cores each (1.17.0 version). I wanted to know if this is something that is expected or if anyone has any pointers regarding this
s
sap1ens
07/10/2023, 4:46 PMI don’t think there is an easy answer.
There are too many factors that are specific to your Flink workload + every major Flink version has a huge number of changes.
You can try gradually upgrading to 1.14, 1.15, etc. to find the version introducing the degradation and then study the changelog for that version.
sap1ens
07/10/2023, 4:46 PMAlso you can always profile your old job, your new job and investigate the bottlenecks you see: https://sap1ens.com/blog/2023/06/04/profiling-apache-flink-applications-with-async-profiler/
s
saqlain pasha
07/10/2023, 5:38 PMThanks @sap1ens for the pointers. Can you please let me know where I can find the changelog? Is there a quick way to find all the PRs that went in during the version upgrade
s
sap1ens
07/10/2023, 5:38 PMFlink blog: https://flink.apache.org/posts/
sap1ens
07/10/2023, 5:38 PMEach release announcement has a pretty detailed list of JIRA issues.
s
saqlain pasha
07/10/2023, 5:40 PMThanks again !
🙌 1
2 Views