Hello folks, I'm trying to understand how Flink's streaming WindowOperator can be deterministic when

allowedLateness

is specified. Say we are doing event time based processing. The initial watermarks are generated at the source. And then every operator computes its watermark as well. In case when there are parallel streams (or probably even multiple Kafka partitions to read from?), the watermark is the minimum across all inputs. If one of the parallel streams is paused (for whatever reason), the watermark would not advance and there would be no late events. But if the parallel stream was not paused, the watermark would be advancing and there would be some late events. So how is event processing deterministic in this case? Btw my understanding of Flink is limited, so some assumptions I made above about how Flink works could be wrong.

Flink is not deterministic in all situations, even if event time is used. The non-determinism comes from situations where processing time is mixed with event time, as occurs with periodic watermarking and with idleness detection. With both periodic watermarking and idleness detection, exactly which events will be marked as late is non-deterministic.

I see! Periodic watermarking could lead to non-deterministic behavior. What if we were using

BoundedOutOfOrdernessWatermarks

in the above scenario to remove the non-determinism in generating watermarks?

Bounded-out-of-orderness is an example of a periodic, non-deterministic watermarking strategy. The problem is that it only periodically generates a new watermark (every 200 ms, measured with wall clock time), which means that some events that would have been late if every conceivable watermark had been generated won't actually be late. Only by generating watermarks after every event -- which is a bad idea from a performance perspective -- can you avoid this.

Ah, ok. Didn't realize

BoundedOutOfOrdernessWatermarks

also emits watermarks periodically. Thank you for the above response, that makes sense. One more follow up question. So can we really achieve determinism even if we emit watermark after every event? It seems to me that even that may not be enough to achieve determinism as different partitions would still be consumed at different speeds. The watermark WindowOperator uses (minimum watermark across all partitions) to determine if an event is late still seems to depend upon the runtime state of the job. Right?

Yes. I think you can only achieve deterministic behavior if you manage to completely avoid late events and idle sources -- which can't be guaranteed.

Perfect, thanks for confirming that David! Appreciate your help with this!