https://pinot.apache.org/ logo
p

Pradeep

09/29/2020, 11:14 PM
Hi, I added some additional columns to star-tree index, but I don’t see the new column being part of the start tree index in the new segments doesn’t modifications get reflected in the new segments? IIRC reloads doesn’t help with modifications to the star tree index, correct me if I am wrong
j

Jackie

09/29/2020, 11:16 PM
Yes, reload cannot modify the existing star-tree, it can only add new star-tree if no one exists yet
p

Pradeep

09/29/2020, 11:16 PM
once the index config is applied, shouldn’t the new segments use the updated index config?
j

Jackie

09/29/2020, 11:17 PM
Yes, the new segments should be able to pick up the new config
p

Pradeep

09/29/2020, 11:19 PM
hmm, weird I don’t see it happening in the newer segments, let me double check
j

Jackie

09/29/2020, 11:19 PM
For realtime table, the new config will only be picked up by the next consuming segment
The current consuming segment will still use the old table config
p

Pradeep

09/29/2020, 11:24 PM
I don’t see it in the newer segments too, I enabled this around midnight yesterday. There are a lot of segments that got generated after that, but I don’t see it in any of those
Is there a scenario where a column will be skipped by star-tree?
if the cardinality of the column is more than
max.leaf.records
then the column wouldn’t be included in the index?
j

Jackie

09/29/2020, 11:36 PM
No, star-tree should not skip any column explicitly configured
What is the old and new star-tree config?
You may search the log for
Starting building star-tree with config: ...
which should print the star-tree configs used
p

Pradeep

09/29/2020, 11:39 PM
let me check that