Radu Stoian
03/29/2023, 6:58 PMmyValueState.value(), inside a KeyedProcessFunction, sometimes returns null despite the fact that the logic in the code should guarantee that the object returned by .value() is not null. These nulls are returned rarely, and do not occur again when the app is restarted and run on the same data that it previously failed on. Note: myValueState is of type ValueState<java.time.LocalDateTime>.
More Context
• I am using Flink 1.15.2, hosted on AWS Kinesis Data Analytics; this is where the error occurs
• The error does not occur locally
• RocksDB is used as the state store backend on AWS Kinesis Data Analytics
Code
• Near the top of my process function, I run the first function for every single event to update my minTimestamp value state
• This function, to my mind, should ensure that the value of this value state should never be null
• Later on in the code I call minTimestamp.value(), which will return null once in a while
void updateMinTimestamp(LocalDateTime newTimestamp) {
try {
final LocalDateTime currentMinTimestamp = minTimestamp.value();
if (currentMinTimestamp == null || newTimestamp.isBefore(currentMinTimestamp)) {
minTimestamp.update(newTimestamp);
}
} catch (Exception e) {
throw new FlinkStateAccessException("failed", e);
}
}
Any information regarding why this error is occurring, and how to prevent it, is much appreciated