Apache Pinot

Hi team, if I add an index to my table’s indexing config, what happens to the old segments? Will the new index be created when the segment is loaded during query? Do I need to reload all the segments?

index will not be created on old segments during query. You need to reload for it to be effectve on older segments

<@UDRLN8MAP> Just for my understanding, reloading takes the latest table config and applies it to all the segments? Does it all the segment into memory as well? Can I run into OOO if I have too many segments for my table?

Not exactly latest table config. It can do some things only like new column, derived column, new index, remove index. It cannot change encoding, drop column.

It doesn't bring the whole segment into memory. It only builds the new index, and attaches it to the existing segment's metadata

<@U01DS9A0DS4> do you have a reload recipe that explains what it can and cannot do and how?

Same as transform function, but done on a live table, where old segments get their new column value backfilled from existing Vikings

so to add a derived column:

• Add transform function with new column
• Reload segment
?

What if we add a sorted index on a column? In that case segment needs to be sorted and will it be loaded in memory?