https://pinot.apache.org/ logo
#general
Title
# general
a

Alice

03/30/2022, 7:52 AM
when I run an explain plan for query multiply times, I got different query results. And the totalDocs number is different. A very simple query like explain plan for select * from table_name. Any idea why is that?
r

Richard Startin

03/30/2022, 8:47 AM
hi can you share the plans?
m

Mayank

03/30/2022, 10:15 PM
also cc: @User
a

Alice

03/31/2022, 3:06 AM
message has been deleted
“replicasPerPartition”: “2"
v

Vivek Iyer Vaidyanathan

03/31/2022, 5:49 AM
Can you please share explain plan output as well? Do you see same explain plan result in all cases but with number of docs being different in “FILTER_MATCH_ENTIRE_SEGMENT(docs:xyz)” line of explain plan?
a

Alice

03/31/2022, 6:02 AM
one output is something like no matching segement.
totalDocs, I mean the one listed here
a

Amrish Lal

03/31/2022, 3:23 PM
Its difficult to say without seeing the actual explain plan outputs and comparing them, but the differences can probably be explained by the following (more details at https://docs.pinot.apache.org/users/user-guide-query/explain-plan):
Copy code
EXPLAIN PLAN output should only be used for informational purposes because it is likely to change from version to version as Pinot is further developed and enhanced.
...
Note that EXPLAIN PLAN output currently extrapolates the overall query plan from a randomly picked Segment on a randomly picked Server, so it is possible that all segments of all servers may not execute the query in exactly the same way. However, in general, the EXPLAIN PLAN output is representative of overall query execution in Pinot.
a

Alice

04/01/2022, 2:12 AM
@User, I tried a new realtime table an here is the output.
a

Amrish Lal

04/01/2022, 5:23 PM
Seems like planner decided to use startree index in one segment, but not the other. Such differences can come up in the explain plan depending upon which host/segment was used.
m

Mayank

04/01/2022, 5:55 PM
Is this documented? If not, could we do so, otherwise it becomes a bit confusing for users.
a

Amrish Lal

04/01/2022, 7:48 PM
Yes, this is how we worded this:
Copy code
Note that EXPLAIN PLAN output currently extrapolates the overall query plan from a randomly picked Segment on a randomly picked Server, so it is possible that all segments of all servers may not execute the query in exactly the same way. However, in general, the EXPLAIN PLAN output is representative of overall query execution in Pinot.
m

Mayank

04/01/2022, 8:33 PM
How can we make it more obvious to the user? In this case it wasn’t I am guessing?
a

Amrish Lal

04/01/2022, 9:44 PM
How about this:
Copy code
Note that EXPLAIN PLAN output shows how the query will run on one randomly picked Segment of one randomly picked Server. The query might run slightly differently on another randomly picked Segment and hence you may see slight variation in EXPLAIN PLAN output when running EXPLAIN PLAN statement on the same query repeatedly. In general though, the EXPLAIN PLAN output is a good representation of the overall query execution in Pinot.