@User I had recently added support for blob like string columns in raw forward index. So physically you can still store them and each column value can go upto millions of characters. But we saw increased heap overhead in the segment generation code path since until it makes into the raw index, the values will be on heap. Secondly, this was used for text search filter so we never selected them and just used for filtering so not sure what the overhead is going to look like on selection path.

Thanks for the detailed explanation, Sidd. Any PR that shows these changes?

@Buchi Reddy https://github.com/apache/incubator-pinot/pull/5256

ok Thank you