I have a basic question on Pinot search (I am sure...
# general
h
I have a basic question on Pinot search (I am sure Pinot will have it - Just want to know internals of how it does it) - suppose I have thousands of segment files over long time. How does a query avoids looking at all these files to give a query result- e.g. if i query for a data for time range. Does Pinot know which segment files to read to fulfil this query. Any detailed doc will help which can explain this..
k
@User
f
I will give you what I’ve understood 😉 Segment can be seen as partition and each segment have his own indexes to provide fast and reliable response. In addition to that segment pruning can be added to pre-filters witch segment to query based on a query filters predicates you time rage for exemple. Pinot expert will give you more detail from the inside of course 🙂 Still a newbie
thankyou 1
r
there's lots to say about this topic but the keyword to search for is "segment pruning" - best to start here https://docs.pinot.apache.org/operators/operating-pinot/tuning/routing
thankyou 1