Hi all, I have a Flink error and was hoping someon...
# troubleshooting
r
Hi all, I have a Flink error and was hoping someone could help I am encountering an error in my Flink app where calling
myValueState.value()
, inside a
KeyedProcessFunction
, sometimes returns
null
despite the fact that the logic in the code should guarantee that the object returned by
.value()
is not null. These nulls are returned rarely, and do not occur again when the app is restarted and run on the same data that it previously failed on. Note:
myValueState
is of type
ValueState<java.time.LocalDateTime>
. More Context • I am using Flink 1.15.2, hosted on AWS Kinesis Data Analytics; this is where the error occurs • The error does not occur locally • RocksDB is used as the state store backend on AWS Kinesis Data Analytics Code • Near the top of my process function, I run the first function for every single event to update my minTimestamp value state • This function, to my mind, should ensure that the value of this value state should never be null • Later on in the code I call
minTimestamp.value()
, which will return null once in a while
Copy code
void updateMinTimestamp(LocalDateTime newTimestamp) {
    try {
        final LocalDateTime currentMinTimestamp = minTimestamp.value();
        if (currentMinTimestamp == null || newTimestamp.isBefore(currentMinTimestamp)) {
            minTimestamp.update(newTimestamp);
        }
    } catch (Exception e) {
        throw new FlinkStateAccessException("failed", e);
    }
}
Any information regarding why this error is occurring, and how to prevent it, is much appreciated