Francis Altomare
09/18/2025, 6:27 PMKafkaPartitionSplit.COMMITTED_OFFSET.
In some cases, it looks like Flink falls back to the Kafka broker’s committed offsets, even though offsets are already available in Flink’s state store. We’ve observed data loss in some of our stateful jobs after restoring from a savepoint. Our best guess as to what’s happening is that the broker’s committed offsets may be ahead of the state offsets.
Question:
• Does anyone know if there are cases when Flink could revert to broker committed offsets even though it has offsets in its state store?
• If yes, under what circumstances does the Kafka source revert to the broker’s committed offsets instead of using the offsets from Flink state?
• Is there a recommended way to ensure Flink always prefers state offsets to avoid data loss after restore?
Thanks a lot for your help!Mohammed Salman Ali Pary
09/22/2025, 1:08 PM