Good morning folks, I'm trying to use Scala with F...
# troubleshooting
r
Good morning folks, I'm trying to use Scala with Flink 1.17.1 locally with docker compose. I've made a simple job and packed it as fat jar and while executing I run into the classic NoSuchMethod error for the scala collection functions. I already checked the built jar for the classpaths and it does contain them, so I'm pretty clueless right now, what else it could be. Well, I also tried to fix that error by adding the scala library jar to the
lib
folder of flink, rebuilding the image and restarting all containers I have but that also didn't resolve it. Is there something simple I'm missing here?
g
I guess there ist still the flink original Scala library in the lib directory. You have to remove it. Take a look at https://flink.apache.org/2022/02/22/scala-free-in-one-fifteen/ Günter
r
damn, that must be it. I was missing on that tiny detail, that I apparently have to remove them myself manually with the detail that beginning with 1.15.x it's possible to do that without crashing Flink...
lemme just check on my local version
damn, that seems to be it! Thank you @guenter hipler!!
saving my day mate 😄
TL;DR I thought that all scala libraries are now removed from flink but now I know that no strict dependency is anymore on them but they are still placed by default into a default flink installation😄
g
You are welcome! Do you know the flink Scala api wrapper https://github.com/flink-extended/flink-scala-api ? It allows using the streaming api with any new Scala Version. Even 3.x Günter
r
no I wasn't aware of that one yet, only the other one 🤔
great, my job runs now after removing it 😄 perfect
g
Examples using the wrapper library https://gitlab.com/scala26/flink/scalawrapper/rtjvm_flink_essentials_scala_wrapper/-/tree/scala_v3?ref_type=heads based on this Tutorial https://rockthejvm.com/p/flink Just a quick proof of concept Günter