using Flink Operator, I often get this kind of err...
# troubleshooting
e
using Flink Operator, I often get this kind of error when submitting a job:
Copy code
Caused by: org.apache.flink.runtime.rest.handler.RestHandlerException: Jar file /tmp/flink-web-efe42546-a97a-4393-87c2-f61f782595dc/flink-web-upload/5d8e11e0-4736-49b7-ac82-0e2bc9bd3b5b_beam.jar does not exist
Somehow the job ends up working after some time when the jar is found. However using Beam, the Beam job fails because of this, and I see the Beam job restarting many, many times before it finally works. Is there any kind of setting to define how to ‘wait’ somehow for the jar to be present before attempting to launch the job? or is it a bug?
I see the problem stems from the fact that I run a HA cluster, but the k8s service load balances between the 3 nodes… so effectively the jar is uploaded on a node, and the run submit call for this jar may end up on a different node and fail, because the jar is not present on that node. If we’re lucky it ends up on the same node. by setting the sessionAffinity: ClientIP on the rest service, this fixes the problem. I filed https://issues.apache.org/jira/browse/FLINK-31509