Boice Lu
09/26/2025, 2:00 AMFlinkSessionJob that consumes data from a Kafka topic and writes it to a BigQuery table.
When the JobManager starts, it seems that the initialization step — possibly checking the BigQuery table — takes a long time. During this delay, the web upload temporary file appears to be deleted, causing the job to fail.
We are seeing the following exception:
Caused by: java.io.FileNotFoundException:
/tmp/flink-web-11008d54-f3d5-4458-8b2c-531d6c88ab1a/flink-web-upload/edc628df-da1b-4b0f-b508-88b3d9891edb_flink-datastream-1.0.0.jar
(No such file or directory)
Has anyone encountered this issue before?
Is there a recommended way to prevent the uploaded jar file from being deleted during job initialization?
Thanks in advance for any help! 🙏Anh Nguyen
09/26/2025, 5:30 PM/tmp/flink-web-11008d54-f3d5-4458-8b2c-531d6c88ab1a/flink-web-upload/edc628df-da1b-4b0f-b508-88b3d9891edb_flink-datastream-1.0.0.jar inside the jobmanager container?Boice Lu
09/30/2025, 4:56 AMAnh Nguyen
10/05/2025, 9:55 PMcurl -X POST -H "Expect:" -F "jarfile=@/Users/flink-datastream-1.0.0.jar" <http://localhost:8081/jars/upload>
the response is probably: edc628df-da1b-4b0f-b508-88b3d9891edb_flink-datastream-1.0.0.jar and I assumed you also saw the jar file in jobManager /tmp/flink-web-11008d54-f3d5-4458-8b2c-531d6c88ab1a/flink-web-upload/edc628df-da1b-4b0f-b508-88b3d9891edb_flink-datastream-1.0.0.jar
Next, you send another Flink API to starts the job. For example,
curl -X POST <http://localhost:8081/jars/edc628df-da1b-4b0f-b508-88b3d9891edb_flink-datastream-1.0.0.jar/run>
After the job got started (RUNNING state), it run the BigQuery as an initialization step, but it took long time to finish. However, during this step, the /tmp/flink-web-11008d54-f3d5-4458-8b2c-531d6c88ab1a/flink-web-upload/edc628df-da1b-4b0f-b508-88b3d9891edb_flink-datastream-1.0.0.jar got deleted inside your JobManager, resulting in FAIL state in the job.
Please correct me if I am wrong in any stepBoice Lu
10/07/2025, 2:33 AMFlinkSessionJob should be equivalent to what you described — it also uploads the JAR to the JobManager and triggers job execution internally.
Tanks for your confirmationAnh Nguyen
10/10/2025, 5:15 PMBoice Lu
10/12/2025, 7:44 AM