Hi all, I am interested in adding some User Define...
# general
e
Hi all, I am interested in adding some User Defined Aggregation Functions (https://docs.pinot.apache.org/developers/developers-and-contributors/extending-pinot/custom-aggregation-function). It looks like you need to recompile from source, rather than drop them in as a jar. "As of today, this requires code change in Pinot but we plan to add the ability to plugin Functions without having to change Pinot code." Does anyone know if work is going on upstream to fix this? If anyone has a ticket they can point me to, that would be awesome. I didn't see anything.
m
Stateless transform/filter functions can be dropped in as jars. There isn't any work going on for stateful aggregation functions
e
Thanks @Mayank. From a naive point of view, it seems like a simple fix. Do you happen to know why it isn't being looked at? Does it need more of an architectural change? The reason I'm asking is because I was hoping for me to gauge how much work it would be to do myself and contribute.
m
I think it is because aggregation functions are not stateless, they bring in state, code dependency as well as require knowledge of what block by block iteration means for example. Stateless filter/transform functions on the other hand don't have that problem
Also, dropping jars can in production deployments is generally considered risky, esp if one doesn't control what is going in there