https://pinot.apache.org/ logo
#general
Title
# general
c

Carl

08/03/2021, 2:18 AM
We have a offline table field truncating texts as it exceed the default 512 length. Is it possible to change the default maxlength without having to reingest all the records to Pinot?
s

Sidd

08/03/2021, 2:28 AM
schema can be updated with configured default length and it will be picked up for new segments. However, it won't change anything for existing segments since the extra text was thrown away
c

Carl

08/03/2021, 2:41 AM
Thanks, I tried to add maxLength to the field by editing schema in controller UI, but it retired as “backward incompatible schema, only allow adding new columns”. Do we have to recreate the schema and table?
Is there currently a configuration in Pinot to override the maxlength to bigger value than 512?
m

Mayank

08/03/2021, 2:50 AM
Yes, you can specify that in the field spec in the schema
c

Carl

08/03/2021, 2:51 AM
Yes but I mean is there a cluster wide configuration to bump up the default maxlength?
m

Mayank

08/03/2021, 3:39 AM
You should be able to bump up the max length on a per table basis. Not sure why you got the error (perhaps you had backward incompatible change other than the max length change?) this is a per column setting and cannot be applied across columns or tables in the cluster
c

Carl

08/03/2021, 3:47 AM
I am only adding “maxLength”:5120 or “maxLenth”:”5120” to one string column from the controller web UI “edit schema”. Would other Pinot API like CLI have less restrictions to edit schema?