`As a general rule, operators are required to comp...
# troubleshooting
s
As a general rule, operators are required to completely process a given watermark before forwarding it downstream. For example, WindowOperator will first evaluate all windows that should be fired, and only after producing all of the output triggered by the watermark will the watermark itself be sent downstream. In other words, all elements produced due to occurrence of a watermark will be emitted before the watermark.
Why do we need to send the watermark downstream? Can anyone explain the exact meaning of this paragraph? TIA
d
Watermarks are typically created very early in the job graph (e.g., in the sources), but they may be needed at any later point in the job graph. E.g., consider a job that has 1-second-long windows in front of 10-second-long windows. Both layers of windowing will rely on watermarks to close their windows. Since watermarks are a measure of stream completeness, they must not be emitted (or forwarded) too early.
s
Cool understood so it boils down to the basics of watermarks
d
Yes. And the paragraph you quoted is really only relevant to folks implementing operators. As a user, you can just expect this to work.
flink 1
s
Cool