Hello Flink community! Quick question about the u...
# troubleshooting
s
Hello Flink community! Quick question about the use of `keyBy`: I have a stream of events with keys that are unique, and I need to process them based on the key -- exactly the use case for
keyBy
. However, the keys change every day, and new records appear in the stream with new keys -- and keys from the previous days will not be repeated. So if I use
keyBy
, it will create a sub-stream which won't see any active traffic in the following days. My question is: what happens to "inactive" (sub-) streams created by
keyBy
? Do they live forever and consume resources? Is there a way to terminate them? And finally, is there an alternate way to process the events in this situation where the keys are changing every day? Thank you.
m
there is TTL for state but it depends on what you do with that keyed stream https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/state/#keyed-datastream
s
Thanks - I'll check that out. I don't need to retain that keyed stream -- once the events for that key are processed, I'd like to "delete" the stream (it will not be needed any more).
m