https://pinot.apache.org/ logo
#general
Title
# general
s

Sidd

05/16/2020, 2:56 PM
@User I had recently added support for blob like string columns in raw forward index. So physically you can still store them and each column value can go upto millions of characters. But we saw increased heap overhead in the segment generation code path since until it makes into the raw index, the values will be on heap. Secondly, this was used for text search filter so we never selected them and just used for filtering so not sure what the overhead is going to look like on selection path.
b

Buchi Reddy

05/16/2020, 10:50 PM
Thanks for the detailed explanation, Sidd. Any PR that shows these changes?
s

Sidd

05/17/2020, 7:21 AM
Support was added as part of above 2 PRs
b

Buchi Reddy

05/17/2020, 10:25 PM
ok Thank you