I am running Flink Job on Kubernetes with Flink Op...
# troubleshooting
s
I am running Flink Job on Kubernetes with Flink Operator. I have to copy a dependency jar to all JM and TM pods inside flink/opt/. So, if we download the jar in flink/opt for jobmanager pod only. Does jobm manager copies the jars in flink/opt/ to taskmanager by default?
s
Yes. Because the same image is used for JM and TMs, jar will be available in both pods
s
No I am not putting the dependency in image. We are downloading the jar from gcs (GCP) location using initContainer. So the jar is downloaded in flink/opt whenever a pod initiates.
s
can you create a new image with all you dependencies and submit a flinkApplication using the custom image ? i assume that jars you are downloading from gcp using initContainer can also be done while creating an image
s
No we don't want to make it part of image. Since these jars will be maintained by Platform and updated frequently. Now users who are utilizing the platform should not rebuild the image if the dependency is updated. Just a job redeployment should be done from users end
So we are downloading the jar in initContainer where the latest version will come from the configMap that platform maintains
I am now trying to pass the jar like this:
Copy code
job:
  jarURI: local:///opt/flink/usrlib/jobs-1.0-SNAPSHOT.jar
  parallelism: 5
  upgradeMode: stateless
  state: running
  args: ["--jars gs://flame-poc/image-build/image/runners-0.0.2-SNAPSHOT-all.jar"]
  entryClass: com.fdp.replication.DQ.FlameReplicationDQJob
using the args of JobSpec, but it is also not working. It is not getting added to classpath of java command on the JM. @Gyula Fóra Can you suggest something here?