We read recently that Groovy may be insecure,

Be careful when you’re using Groovy, with the recent log4J security is definitely important, make sure you know what you’re doing with that feature. I would suggest turning it off if you don’t really need that feature, although it’s very useful because it helps in terms of just writing your own custom function without having to understand Pinot internal details. And once you get the functionality, then you end up going and writing your own UDF.

(https://www.startree.ai/blogs/apache-pinot-2021-recap-and-2022-roadmap) Is it now recommended to avoid Groovy for ingestion transforms, or perhaps there are certain patterns one should avoid for security reasons?

I think what the transcript is stating is that Groovy functions provide a good way to prototype a functionality, which once validated, can be turned into a UDF.

few things about groovy, • we do disable do system level stuff in the script. These functions have limited access. • The threat vectors are from two things 1. groovy function 2. the data on which the groovy function operates on. So, in terms of security assessment - you will have to look into who can write these functions and are the functions written in a way that users can control/modify the behavior by passing in malformed data.

Any chance you could provide a concrete example? e.g. Groovy({…}, …) = bad

Check dm 😃