xiaohe lan
01/11/2023, 5:45 PMdocker-compose ps
NAME COMMAND SERVICE STATUS PORTS
operations-playground-clickevent-generator-1 "/docker-entrypoint.…" clickevent-generator running 8081/tcp
operations-playground-client-1 "/docker-entrypoint.…" client exited (1)
operations-playground-jobmanager-1 "/docker-entrypoint.…" jobmanager running 0.0.0.0:8081->8081/tcp, :::8081->8081/tcp
operations-playground-kafka-1 "start-kafka.sh" kafka running 0.0.0.0:9092->9092/tcp, 0.0.0.0:9094->9094/tcp, :::9092->9092/tcp, :::9094->9094/tcp
operations-playground-taskmanager-1 "/docker-entrypoint.…" taskmanager running 8081/tcp
operations-playground-zookeeper-1 "/bin/sh -c '/usr/sb…" zookeeper running 3888/tcp
When I check the exceptions I see the following:
org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster.
at org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.CompletionException: org.apache.flink.util.FlinkRuntimeException: Failed to create checkpoint storage at checkpoint coordinator side.
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source)
... 4 more
Caused by: org.apache.flink.util.FlinkRuntimeException: Failed to create checkpoint storage at checkpoint coordinator side.
at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:337)
at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:245)
at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.enableCheckpointing(DefaultExecutionGraph.java:511)
at org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder.buildGraph(DefaultExecutionGraphBuilder.java:317)
at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:156)
at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:361)
at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:206)
at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:134)
at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:152)
at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:119)
at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:369)
at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:346)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:123)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:95)
at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
... 4 more
Caused by: java.io.IOException: Failed to create directory for shared state: file:/tmp/flink-checkpoints-directory/84da55bc3e4f0429bf874b80c498b94c/shared
at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess.initializeBaseLocationsForCheckpoint(FsCheckpointStorageAccess.java:117)
at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:333)
... 18 more
I do have the directory created already in the host:
ls /tmp/flink*
/tmp/flink-checkpoints-directory:
/tmp/flink-savepoints-directory:
Any idea what is going on here ?xiaohe lan
01/11/2023, 6:30 PMls -ld /tmp/flink*
drwxr-xr-x 2 xilan staff 64 Jan 10 22:36 /tmp/flink-checkpoints-directory
drwxr-xr-x 2 xilan staff 64 Jan 10 22:36 /tmp/flink-savepoints-directory
in the container:
root@b4776be72d15:/opt/flink# ls -ld /tmp/flink-savepoints-directory /tmp/flink-checkpoints-directory
drwxr-xr-x 3 root root 4096 Jan 11 17:37 /tmp/flink-checkpoints-directory
drwxr-xr-x 2 root root 4096 Jan 11 06:17 /tmp/flink-savepoints-directory
They have different owners/groups in the host and the container, is this OK ?