I think I have just the right case for using Flink...
# randomd
Daniel Hristov
04/26/2023, 2:45 PMI think I have just the right case for using Flink Queryable State, but am a bit concerned about the documentation stating that it’s an unstable API. I don’t really mind having to refactor some client code if the API changes, but are there any long-term plans to support and develop the mechanism. Any concerns (performance, resources, etc) to have in mind?
m
Martijn Visser
04/26/2023, 2:46 PMI think the biggest concern is that it’s considered deprecated by the Flink community 😅
d
Daniel Hristov
04/26/2023, 4:51 PMOh, this pretty much disqualifies it for me
Daniel Hristov
04/26/2023, 4:52 PMThanks for letting me know
m
Martijn Visser
04/26/2023, 4:53 PMNo problem
Martijn Visser
04/26/2023, 4:53 PMI would be interested in understand your use case though 🙂
d
Daniel Hristov
04/26/2023, 5:52 PMWe have a stream ingesting data, accumulating some data in keyed state and ultimately flushing this state to an ES sink (so that it’s searchable)
Daniel Hristov
04/26/2023, 5:58 PMThere’s another flow of different data that needs to be enriched with the results from the first one. Right now this second stream is looking up data written by the ES sink, but I thought it could be easier if it could just query the state (the lookup key happens to be compatible to the one used to partition the data in the first stream)
Daniel Hristov
04/27/2023, 5:20 PMDo you have any clues if the Queryable state is supposed to be replaced by something with similar functionality (allowing to query for internal data in the state backend without outputting it to a sink) or just be discontinued?
m
Martijn Visser
04/27/2023, 6:44 PMI recall there was a discussion thread a while ago on this
Martijn Visser
04/27/2023, 6:45 PMWould need to look it up