Hello! I am trying to submit a Flink SQL job that uses a custom python udf. I can get this to work locally, but when I dockerize it with the official flink image, I get the following exception:

Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.runtime.operators.python.scalar.PythonScalarFunctionOperator

any ideas why the class is not loading? Is there another JAR that I need to include to get this to work?

i had same error several weeks ago (pyflink + udf + aws emr) In my case, problem was resolved by removing pipelne.jars setting. For emr flink, it takes libs straight from lib folder, no need in this setting at all. Check if its relevant for you

Yup, that did the trick, I removed the jars argument specifier and moved the added jars to

lib

and it worked. Many thanks!