Hello, Can someone please confirm if we can apply star-tree indexing to the realtime tables? Below article says "star-tree indexing can't be applied due to the nature for data for realtime tables" https://medium.com/@surajkmth29/apache-pinot-tables-and-segments-a72dc5854876

Can't seem to find your quote in this post. But the StarTree index can be applied to realtime tables, with the exception of realtime tables with upsert enabled.

Hey @robert zych, thanks for responding, you can find the statement in the Performance row in below image from the article https://miro.medium.com/max/1100/1*Y0Lr9HxcNssxE7q_ddJZDw.webp

To be more accurate, star-tree index cannot be applied to the consuming segments (mutable). It can be applied to the sealed segments (immutable) in the real-time table.

Are you saying that the StarTree index is updated whenever a consuming segment closes?

Yes, correct. It is created when the segment seals

This means that how far behind the StarTree index is to the latest offset is determined by the ingestion rate and the table’s time and space settings.

Most indexes have one immutable version and one mutable version. When consuming segment seals, pinot will generate the immutable version of the index. Currently the following index types doesn't have mutable version: range index, bloom filter, star-tree index

Thanks for the explanation! When Pinot executes a query that requires data from both the immutable and mutable segments and there is no mutable version of the index (eg star-tree) does it scan the data from the consuming/mutable segment and combine it with the results from the index?