Hi there! I’m running flink1.17 on kubernetes(oper...
# troubleshooting
a
Hi there! I’m running flink1.17 on kubernetes(operator v1.5). The simplest possible task to read files from S3. It works perfectly as it should - reads files from folder, if file is changed it takes the latest changes. Unless I restart the job from savepoint/checkpoint. After restart it does not load files which were read already(that’s expected) but at the same time it does not process the new files anymore. It works as expected on amazon EMR, locally in docker(reading same folder on s3), but not on k8s… Somehow related message Which direction I should be looking? 🙏
Untitled.scala
With debug enabled I was able to get the following
Copy code
DEBUG org.apache.flink.streaming.api.functions.source.ContinuousFileMonitoringFunction - Ignoring <s3://my-bucket/my-folder/my.json>, with mod time= 1691069277000 and global mod time= 9223372036854775807
it looks like during saving(restoring from?) a checkpoint, the globalModificationTime is saved(restored?) with Long.MAX_VAL 🤯 in
org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java
Restoring from savepoint happens already with Long.MAX_VAL
Copy code
DEBUG org.apache.flink.streaming.api.functions.source.ContinuousFileMonitoringFunction - ContinuousFileMonitoringFunction retrieved a global mod time of 9223372036854775807.
That means this value is added during savepoint creation