Hello,
We would like to use pulsar as a buffer to process aggregations. In our case, we would like to maximize batch receive size instead of latency, ie: BatchReceivePolicy with a very long timeout and a high number of messages.
I've read the client source code, and in my undestanding, the client is performing batching in memory (with incomingMessages blocking queue).
Since my application will connect to a lot of topics (possibly 1-5k), I'm not comfortable with this. I'd like to prevent OOM when there is a lot of pending messages. Since pulsar broker knows how much messages are in backlog, is there a way to have a callback to notify client that it can consume this topic ?
Thanks for your feedbacks