Hello, I have a question about Flink Sliding Window and aggregation function. We have a job which calculate max/min/sum in the last 24 hours(event time) for each id in a given stream. Our approach is using sliding event window in a keyed stream. But as the data grows we found that sliding window consumed a lot of disk space because the states is so large. So we add aggregate functions to the job. But after adding the agg function, we found that the watermark generation seems stops and the job seems stopped.
The code is something like this
keyedStream.window(SlidingEventTimeWindows.of(Time.minutes(1), Time.seconds(5)))
.aggregate(new Last24HAggregator(), new MyProcessWindowFunction())
.addSink(new MySink());
Before we add the aggregation function, everything seems fine. My guess is that maybe aggregation function is not compatible with event time processing semantics.