Kirill Ternovsky
05/31/2024, 5:27 PMFileSource using an s3 path. I've enabled `flink-s3-fs-hadoop`(getting past the "no implementation for 's3'" issue), but during startup I see:
jobmanager-1 | 2024-05-31 17:25:51,270 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting StandaloneApplicationClusterEntryP
oint (Version: 1.19.0, Scala: 2.12, Rev:eaffd22, Date:2024-03-06T15:18:44+01:00)
jobmanager-1 | 2024-05-31 17:25:51,270 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS current user: flink
jobmanager-1 | 2024-05-31 17:25:51,270 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Current Hadoop/Kerberos user: <no hadoop dependency found>
And the jobmanager just hangs after initializing the job:
jobmanager-1 | 2024-05-31 17:25:53,098 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Starting execution of job 'TEST READ' (38bae99cbfab3d01bc2c80d00289a3f1) under job master id 00000000000000000000000000000000.
jobmanager-1 | 2024-05-31 17:25:53,101 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Connecting to ResourceManager pekko.tcp://flink@jobmanager:6123/user/rpc/resourcemanager_*(00000000000000000000000000000000)
jobmanager-1 | 2024-05-31 17:25:53,103 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Resolved ResourceManager address, beginning registration
jobmanager-1 | 2024-05-31 17:25:53,104 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering job manager 00000000000000000000000000000000@pekko.tcp://flink@jobmanager:6123/user/rpc/jobmanager_2 for job 38bae99cbfab3d01bc2c80d00289a3f1.
jobmanager-1 | Job has been submitted with JobID 38bae99cbfab3d01bc2c80d00289a3f1
jobmanager-1 | 2024-05-31 17:25:53,108 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager 00000000000000000000000000000000@pekko.tcp://flink@jobmanager:6123/user/rpc/jobmanager_2 for job 38bae99cbfab3d01bc2c80d00289a3f1.
jobmanager-1 | 2024-05-31 17:25:53,109 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - JobManager successfully registered at ResourceManager, leader id: 00000000000000000000000000000000.
jobmanager-1 | 2024-05-31 17:25:53,130 INFO org.apache.flink.fs.s3.common.token.AbstractS3DelegationTokenReceiver [] - Updating Hadoop configuration
jobmanager-1 | 2024-05-31 17:25:53,131 INFO org.apache.flink.fs.s3.common.token.AbstractS3DelegationTokenReceiver [] - Updated Hadoop configuration successfully
jobmanager-1 | 2024-05-31 17:25:53,191 WARN org.apache.hadoop.metrics2.impl.MetricsConfig [] - Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
jobmanager-1 | 2024-05-31 17:25:53,198 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl [] - Scheduled Metric snapshot period at 10 second(s).
jobmanager-1 | 2024-05-31 17:25:53,198 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl [] - s3a-file-system metrics system started
jobmanager-1 | 2024-05-31 17:25:53,210 WARN org.apache.hadoop.util.NativeCodeLoader [] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
There's no further output.