https://pinot.apache.org/ logo
Join Slack
Powered by
Can someone please point me to documentation for `...
# getting-started
p
Can someone please point me to documentation for 
enableDefaultStarTree
and 
enableDynamicStarTreeCreation
fields in the table confi? I want to understand what does a default / dynamic star-tree index mean.
n
Dynamic: https://docs.pinot.apache.org/configuration-reference/table#table-index-config (search for that field here). If this is not set, startree wont get added/altered on reload segmets call
p
hmm i somehow missed the default index generation part when i read about star-tree index. thank you for sharing these. i did read the table index config one. Please correct me if I am wrong - setting dynamic star-tree to true will re-generate star-tree index after table config is updated and we reload segments?
n
when you reload segments - yes
p
regardless of whether that property is set to true or false? or only if that property is set to true?
let me just rephrase - do your recommend setting it to true or false?
i am basically trying to figure out answer to this question - https://apache-pinot.slack.com/archives/C01H1S9J5BJ/p1636420457030100
n
only if you set it to true 🙂
you must set it to true if you expectbto make changes to the startree and would like those reflected in the index inside the segments
p
Got it. Thanks Neha.
Asking a kinda obvious question - schema change which doesn't impact change in star-tree index doesn't warrant a reload segments call? and everything should just work seamlessly?
n
are you adding a new column? typically, after you add a new column, you would want to do a reloadAll segments. That way the new column gets populated into the existing segments (as default value)
if you dont do a reload, you will have old segments with one less column than the new segments. This results in a mismatch in the schema, when a query spans across old and new segments
a query like 
select *…
will error out, because the columns fetched from old and new will be differet.
before you do a reloadAll, would be a good idea to disable the 
enableDynamicStarTreeIndex
flag in the table . Since your schema change is not affecting the startree, would be good to skip it while reloading to be safe
p
Got it. This is very helpful. Thank you. And yes I was thinking of the case where we are adding a new column. I lied when I said before that we don't have schema registry. We have one - but is not a confluent one and a homegrown one so my option is to either implement my own version of event decoder to use our schema registry or embed schema with in the table config.
Open in Slack
PreviousNext
Powered by