No, this is the number of segments the query had t...
# generalm
Mayank
07/21/2020, 2:29 AM
No, this is the number of segments the query had to process.
b
Buchi Reddy
07/21/2020, 2:30 AMYeah. what does process mean exactly? Can a query process a segment without loading it into mem?
m
Mayank
07/21/2020, 2:30 AM
Segments are always mmapped, they are not loaded/unloaded per query
Mayank
07/21/2020, 2:31 AM
Query can prune out segments looking at the metadata, rest it has to process (look at values)
b
Buchi Reddy
07/21/2020, 2:31 AMWell, if there isn’t enough RAM, real page loading/unloading happens per query right?
m
Mayank
07/21/2020, 2:32 AM
That is OS level paging
Mayank
07/21/2020, 2:32 AM
Not Pinot level
b
Buchi Reddy
07/21/2020, 2:33 AMTrue. So we have to look at OS level metrics if we need to debug any issues around that?
m
Mayank
07/21/2020, 2:34 AM
From Pinot JVM perspective all data is available to be read. MMaping implies that OS may load/unload pages as needed (not visible to Pinot JVM).
Mayank
07/21/2020, 2:35 AM
So some pages of a segment may already be available from previous queries
Mayank
07/21/2020, 2:35 AM
Does this make sense
b
Buchi Reddy
07/21/2020, 2:38 AMyeah totally. I didn’t necessarily mean Pinot was loading/unloading but we’re on the same page.
Buchi Reddy
07/21/2020, 2:43 AMWill have to check some OS page loading level metrics to get my question answered
m
Mayank
07/21/2020, 2:44 AM
What is your question