Hi team qq. I have a stream that we want to reduc...
# troubleshooting
m
Hi team qq. I have a stream that we want to reduce over a sliding window by event time. The problem is we want every single time window to be accounted for, even if there's no events. Currently the plan is to use a keyed process function. In the process function we • Maintain a list of "windows". Basically a list of timestamped lists (where each list belongs to a time window) • Any event that comes in gets added to all the windows they belong to • If the event has a new key, then our context has another timer linked to this key • Realistically we'd want every single key to always be there. However we may want to delete keys ◦ Either add in some manual intervention through perhaps a broadcast ◦ Or we just have a TTL for each key state (likely what I'd do) • Can also allow late arrivals under this model I believe this will work for our usecase, it shouldn't be overly daunting to set up. But is there a more ergonomic way to do this? I've looked at how I can override some of the existing Window functionality to pull this off but I don't see a way. It feels like it should be very simple. I want a sliding window where the window creation is decoupled from element input. But there doesn't seem to be a cleaner way to implement this than above^. It'd be nice to use existing window features