Hey folks, :wave: I鈥檓 running looking to migrate a...
# troubleshooting
k
Hey folks, 馃憢 I鈥檓 running looking to migrate a job from 1.15.2 to 1.16.1. It鈥檚 deployed via the Flink Kubernetes Operator (v1.4) and is based on the flink:1.16.1-scala_2.12-java8 image. The job fails to deploy and the job manager logs shows that it failed due to:
Copy code
Caused by: java.net.MalformedURLException: unknown protocol: local
	at java.net.URL.<init>(URL.java:618) ~[?:1.8.0_372]
	at java.net.URL.<init>(URL.java:508) ~[?:1.8.0_372]
	at java.net.URL.<init>(URL.java:457) ~[?:1.8.0_372]
	at org.apache.flink.configuration.ConfigUtils.decodeListFromConfig(ConfigUtils.java:133) ~[flink-dist-1.16.1.jar:1.16.1]
	at org.apache.flink.client.cli.ExecutionConfigAccessor.getJars(ExecutionConfigAccessor.java:77) ~[flink-dist-1.16.1.jar:1.16.1]
	at org.apache.flink.client.deployment.executors.PipelineExecutorUtils.getJobGraph(PipelineExecutorUtils.java:69) ~[flink-dist-1.16.1.jar:1.16.1]
The manifest for the job is largely unchanged, other than the
flinkVersion: v1_16
. The jar config is defined as:
Copy code
job:
    jarURI: local:///opt/flink/usrlib/job.jar
I tried this for both 1.16.1 and 1.16.2. I suspect I鈥檓 just missing something silly here, but I don鈥檛 see any mention of
local
being deprecated or any additional steps to migrate a pipeline from 1.15 -> 1.16. Would anyone have any thoughts about what might be causing this parsing error?
g
This is a FlinkDeployment object right?
local should work, very strange because this hasn鈥檛 changed in recent Flink versions.
can you share the whole CR (yaml)?
k
Thanks for taking a look, Gyula! I managed to stumble on the solution this morning and it looks like it was a misconfiguration on our end. In our application code, as part of the constructing the StreamingExecutionEnvironment, we load the external
conf.yaml
and load it into the environment via
env.configure(conf, env.getClass.getClassLoader)
. Now that the StreamingExecutionEnvironment
configure
method supports overriding
JARS
we were inadvertently clobbering the correctly configured JAR with the
local:///
JAR defined in the YAML file. Tested out just calling
conf.removeConfig(PipelineOptions.JARS)
before setting the env config and the pipeline booted up without issue.