Heyo, I've enabled watermark alignment on my flink apps (1.15.2) running on KDA. We noticed that it seems to pause all processing pretty much straight away and that the watermarkAlignmentDrift metric is super low. Can anyone help me understand what is going on with this?
Felix Angell
05/02/2023, 9:24 AM
i'm guessing it's not weird underflow but just subtracting big watermarks. our alignment is configured at 1s updates with 20s max drift (the default specified on the docs)
u
Urs Schoenenberger
05/02/2023, 11:44 AM
just a guess, but I think it's a plausible one: this value is very close to the minimum for a signed 64bit long, so I assume that Long.MIN_VALUE is used as a watermark if no events have been seen on one operator instance, which causes the drift