Hi Folks, I was exploring Apache Pulsar functions....
# generalc
Chaitanya Gudipati
11/05/2025, 3:48 PMHi Folks, I was exploring Apache Pulsar functions. From the documentation, we seem to use Apache Bookkeeper as the state storage interface.
Couple of questions on the state storage for pulsar functions:
1\ Do we have any upper limit on the storage allocated for the pulsar function state storage?
2\ Do we have a tiered storage paradigm even for the pulsar function state storage similar to the event stream storage?
TIA.
d
David K
11/05/2025, 3:56 PM“Do we have any upper limit on the storage allocated for the pulsar function state storage?” Are you asking about the max size of an object that can be stored on BookKeeper?
David K
11/05/2025, 3:57 PM“Do we have a tiered storage paradigm even for the pulsar function state storage, similar to the event stream storage?” Not at this time. It is best to expand the BookKeeper cluster if more storage is required.
👍 1
c
Chaitanya Gudipati
11/05/2025, 3:57 PMMax object size as well as cumulatively how much data can we store.
d
David K
11/05/2025, 3:58 PMData storage is virtually unlimited, as the Bookeeper layer can be scaled horizontally.
👍 1
David K
11/05/2025, 3:59 PMIn terms of object size, there are probably some practical limits imposed by the resources allocated to the functions themselves, as they would have to read all of the data into memory from the state store. HTH
c
Chaitanya Gudipati
11/05/2025, 4:00 PMI see got it. Thank you.