https://pinot.apache.org/ logo
Join Slack
Powered by
Hi team, Anyone can please explain me, Query 1 tak...
# general
a
Hi team, Anyone can please explain me, Query 1 taking around 120 secs whereas Query 2 taking 15-20 secs to give same result. Total Docs in table 6Billion and my table is RT table Is it because in Query2, all 3 queries are computed parallely? or in first query segments are loaded to memory and then other two query takes very less time hence overall less query time Query 1:
Copy code
SELECT * FROM table
  WHERE customer_id = 1234
    AND msisdn IN ( ..1000 msisdns)
Query 2:
Copy code
SELECT * FROM table
  WHERE customer_id = 1234
    AND msisdn IN ( ..350 msisdns)
  UNION ALL
  SELECT * FROM append_iot_session_events
  WHERE customer_id = 1234
    AND msisdn IN (..350 msisdns)
  UNION ALL
  SELECT * FROM append_iot_session_events
  WHERE customer_id = 1234
    AND msisdn IN (..300 msisdns)
m
There’s an interstage data transfer overhead in the second one, which can be large if data size is huge. Are you using MSQE for both? cc @Gonzalo Ortiz
a
yes i m using MSQE
g
Can you share the stageStats and explain plan of both queries?
(stage stats are included in the response as a json entry)
a
stage stats for query 1
Copy code
"stageStats": {
    "type": "MAILBOX_RECEIVE",
    "executionTimeMs": 1484,
    "emittedRows": 1094,
    "fanIn": 20,
    "rawMessages": 263,
    "deserializedBytes": 711119,
    "deserializationTimeMs": 4,
    "upstreamWaitMs": 15098,
    "children": [
      {
        "type": "MAILBOX_SEND",
        "executionTimeMs": 14936,
        "emittedRows": 1094,
        "stage": 1,
        "parallelism": 20,
        "fanOut": 1,
        "rawMessages": 263,
        "serializedBytes": 706422,
        "serializationTimeMs": 9,
        "children": [
          {
            "type": "LEAF",
            "table": "iot_advance_session_events",
            "executionTimeMs": 30244,
            "emittedRows": 1094,
            "numDocsScanned": 1094,
            "totalDocs": 6605466255,
            "numEntriesScannedInFilter": 45,
            "numEntriesScannedPostFilter": 39384,
            "numSegmentsQueried": 8926,
            "numSegmentsProcessed": 776,
            "numSegmentsMatched": 243,
            "numConsumingSegmentsQueried": 36,
            "minConsumingFreshnessTimeMs": 1761643012601,
            "numSegmentsPrunedByServer": 8150
          }
        ]
      }
    ]
  },
stage stats for query 2
Copy code
"stageStats": {
    "type": "MAILBOX_RECEIVE",
    "executionTimeMs": 8,
    "emittedRows": 1096,
    "fanIn": 35,
    "rawMessages": 992,
    "deserializedBytes": 1818008,
    "deserializationTimeMs": 10,
    "downstreamWaitMs": 9783,
    "upstreamWaitMs": 10716,
    "children": [
      {
        "type": "MAILBOX_SEND",
        "executionTimeMs": 11188,
        "emittedRows": 1096,
        "stage": 1,
        "parallelism": 35,
        "fanOut": 1,
        "rawMessages": 992,
        "serializedBytes": 1804040,
        "serializationTimeMs": 60,
        "children": [
          {
            "type": "UNION",
            "executionTimeMs": 11104,
            "emittedRows": 1096,
            "children": [
              {
                "type": "MAILBOX_RECEIVE",
                "executionTimeMs": 11094,
                "emittedRows": 521,
                "fanIn": 20,
                "inMemoryMessages": 32,
                "rawMessages": 1140,
                "deserializedBytes": 905931,
                "deserializationTimeMs": 33,
                "downstreamWaitMs": 3,
                "upstreamWaitMs": 48136,
                "children": [
                  {
                    "type": "MAILBOX_SEND",
                    "executionTimeMs": 537,
                    "emittedRows": 521,
                    "stage": 2,
                    "parallelism": 20,
                    "fanOut": 35,
                    "inMemoryMessages": 19,
                    "rawMessages": 841,
                    "serializedBytes": 883200,
                    "serializationTimeMs": 45,
                    "children": [
                      {
                        "type": "LEAF",
                        "table": "iot_advance_session_events",
                        "executionTimeMs": 946,
                        "emittedRows": 521,
                        "numDocsScanned": 521,
                        "totalDocs": 6607154314,
                        "numEntriesScannedInFilter": 22,
                        "numEntriesScannedPostFilter": 18756,
                        "numSegmentsQueried": 8928,
                        "numSegmentsProcessed": 777,
                        "numSegmentsMatched": 129,
                        "numConsumingSegmentsQueried": 36,
                        "minConsumingFreshnessTimeMs": 1761643070296,
                        "numSegmentsPrunedByServer": 8151
                      }
                    ]
                  }
                ]
              },
              {
                "type": "MAILBOX_RECEIVE",
                "emittedRows": 575,
                "fanIn": 20,
                "inMemoryMessages": 33,
                "rawMessages": 1152,
                "deserializedBytes": 940373,
                "deserializationTimeMs": 35,
                "downstreamWaitMs": 4930,
                "upstreamWaitMs": 53149,
                "children": [
                  {
                    "type": "MAILBOX_SEND",
                    "executionTimeMs": 526,
                    "emittedRows": 575,
                    "stage": 3,
                    "parallelism": 20,
                    "fanOut": 35,
                    "inMemoryMessages": 26,
                    "rawMessages": 888,
                    "serializedBytes": 919694,
                    "serializationTimeMs": 29,
                    "children": [
                      {
                        "type": "LEAF",
                        "table": "iot_advance_session_events",
                        "executionTimeMs": 975,
                        "emittedRows": 575,
                        "numDocsScanned": 575,
                        "totalDocs": 6607154258,
                        "numEntriesScannedInFilter": 25,
                        "numEntriesScannedPostFilter": 20700,
                        "numSegmentsQueried": 8928,
                        "numSegmentsProcessed": 777,
                        "numSegmentsMatched": 131,
                        "numConsumingSegmentsQueried": 36,
                        "minConsumingFreshnessTimeMs": 1761643070296,
                        "numSegmentsPrunedByServer": 8151
                      }
                    ]
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  },
Plan for query 1
Copy code
Execution Plan
LogicalProject(__key=[$3], __metadata$offset=[$4], __metadata$partition=[$5], __metadata$recordTimestamp=[$6], apn_name=[$7], cell_id=[$8], circle_name=[$9], created_on=[$10], customer_id=[$11], device_ip=[$12], disconn_reason=[$13], dummy_start_rec=[$14], end_time_new=[$15], end_time_old=[$16], event_time_new=[$17], event_time_old=[$18], ggsn_circle=[$19], ggsn_ip=[$20], imei=[$21], in_pkt_new=[$22], in_pkt_old=[$23], is_deleted=[$24], lac_id=[$25], msisdn=[$26], nas_id=[$27], out_pkt_new=[$28], out_pkt_old=[$29], rat_type=[$30], session_id=[$31], sgsn_ip=[$32], start_time_new=[$33], start_time_old=[$34], status_new=[$35], status_old=[$36], updated_on=[$37], version=[$38])
  LogicalFilter(condition=[AND(=($11, 22370), >=($10, 1761555455000), SEARCH($26, Sarg[1000 ...msisdn LIST...))])
    LogicalTableScan(table=[[default, iot_advance_session_upsert]])
Plan for query 2
Copy code
Execution Plan
LogicalUnion(all=[true])
  PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
    LogicalUnion(all=[true])
      PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
        LogicalProject(__key=[$3], __metadata$offset=[$4], __metadata$partition=[$5], __metadata$recordTimestamp=[$6], apn_name=[$7], cell_id=[$8], circle_name=[$9], created_on=[$10], customer_id=[$11], device_ip=[$12], disconn_reason=[$13], dummy_start_rec=[$14], end_time_new=[$15], end_time_old=[$16], event_time_new=[$17], event_time_old=[$18], ggsn_circle=[$19], ggsn_ip=[$20], imei=[$21], in_pkt_new=[$22], in_pkt_old=[$23], is_deleted=[$24], lac_id=[$25], msisdn=[$26], nas_id=[$27], out_pkt_new=[$28], out_pkt_old=[$29], rat_type=[$30], session_id=[$31], sgsn_ip=[$32], start_time_new=[$33], start_time_old=[$34], status_new=[$35], status_old=[$36], updated_on=[$37], version=[$38])
          LogicalFilter(condition=[AND(=($11, 22370), >=($10, 1761382655000), SEARCH($26, Sarg[350 ...msisdn LIST...))])
            LogicalTableScan(table=[[default, iot_advance_session_upsert]])
      PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
        LogicalProject(__key=[$3], __metadata$offset=[$4], __metadata$partition=[$5], __metadata$recordTimestamp=[$6], apn_name=[$7], cell_id=[$8], circle_name=[$9], created_on=[$10], customer_id=[$11], device_ip=[$12], disconn_reason=[$13], dummy_start_rec=[$14], end_time_new=[$15], end_time_old=[$16], event_time_new=[$17], event_time_old=[$18], ggsn_circle=[$19], ggsn_ip=[$20], imei=[$21], in_pkt_new=[$22], in_pkt_old=[$23], is_deleted=[$24], lac_id=[$25], msisdn=[$26], nas_id=[$27], out_pkt_new=[$28], out_pkt_old=[$29], rat_type=[$30], session_id=[$31], sgsn_ip=[$32], start_time_new=[$33], start_time_old=[$34], status_new=[$35], status_old=[$36], updated_on=[$37], version=[$38])
          LogicalFilter(condition=[AND(=($11, 22370), >=($10, 1761382655000), SEARCH($26, Sarg[350 ...msisdn LIST...))])
            LogicalTableScan(table=[[default, iot_advance_session_upsert]])
  PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
    LogicalProject(__key=[$3], __metadata$offset=[$4], __metadata$partition=[$5], __metadata$recordTimestamp=[$6], apn_name=[$7], cell_id=[$8], circle_name=[$9], created_on=[$10], customer_id=[$11], device_ip=[$12], disconn_reason=[$13], dummy_start_rec=[$14], end_time_new=[$15], end_time_old=[$16], event_time_new=[$17], event_time_old=[$18], ggsn_circle=[$19], ggsn_ip=[$20], imei=[$21], in_pkt_new=[$22], in_pkt_old=[$23], is_deleted=[$24], lac_id=[$25], msisdn=[$26], nas_id=[$27], out_pkt_new=[$28], out_pkt_old=[$29], rat_type=[$30], session_id=[$31], sgsn_ip=[$32], start_time_new=[$33], start_time_old=[$34], status_new=[$35], status_old=[$36], updated_on=[$37], version=[$38])
      LogicalFilter(condition=[AND(=($11, 22370), >=($10, 1761382655000), SEARCH($26, Sarg[300 ...msisdn LIST...))])
        LogicalTableScan(table=[[default, iot_advance_session_upsert]])
g
In the stats you can see that the 
numEntriesScannedPostFilter
is high. That is the number of rows that have been filtered after the indexes are applied. Do you have an inverted index on 
msisdn
? That may accelerate your query. I also though the first query was faster than the second, but it seems to be the opposite. If the query with unions is faster, probably the reason is on the one hand what you commented about parallelism (where each subquery is executed concurrently) plus the fact that without an index, that IN (or maybe another predicate you didin’t include, like the = and >= that can be seen in the logical filters of the explain plan) is using full scan, which usually benefits from more parallelism
a
yes i have inverted index on msisdn
g
and what about the other predicates applied? because the explain plan includes more predicates than the one you listed in your SQL queries
a
yes i have one condition AND created_on >= 1761382655000
Copy code
SELECT * FROM table
  WHERE customer_id = 1234
    AND created_on >= 1761382655000
    AND msisdn IN ( ..1000 msisdns)
and
Copy code
SELECT * FROM table
  WHERE customer_id = 1234
    AND created_on >= 1761382655000
    AND msisdn IN ( ..350 msisdns)
  UNION ALL
  SELECT * FROM append_iot_session_events
  WHERE customer_id = 1234
    AND created_on >= 1761382655000
    AND msisdn IN (..350 msisdns)
  UNION ALL
  SELECT * FROM append_iot_session_events
  WHERE customer_id = 1234
    AND created_on >= 1761382655000
    AND msisdn IN (..300 msisdns)
i have also range index on created_on
Copy code
"maxRowsInOperator": 1094,
  "requestId": "551798614000000187",
  "brokerId": "Broker_10.13.63.241_8099",
  "numDocsScanned": 1094,
  "totalDocs": 6605466255,
  "numEntriesScannedInFilter": 45,
  "numEntriesScannedPostFilter": 39384,
  "numServersQueried": 0,
  "numServersResponded": 0,
  "numSegmentsQueried": 8926,
this is stats for query 1 and if do union all two times means 500 msisdns and 500 msisdns docs count and other stats becomes * 2 (no of unions in query)
Copy code
"maxRowsInOperator": 1096,
  "requestId": "580427765000000205",
  "brokerId": "Broker_10.13.63.242_8099",
  "numDocsScanned": 1096,
  "totalDocs": 13214308572,
  "numEntriesScannedInFilter": 47,
  "numEntriesScannedPostFilter": 39456,
  "numServersQueried": 0,
  "numServersResponded": 0,
  "numSegmentsQueried": 17856,
but query2 performs better by alot time diff query 1 time
query 2 time (example of 2 times union) for 3 times union it was more less around 10-15 sec
g
the fact that we have a 
numEntriesScannedPostFilter
means we are not using filters for something. Can you run the explain plan for Q1 in SSE? Or alternatively keep using MSE but use the explain plan asking servers (which can be done by adding 
SET explainAskingServers=true;
as explained in https://docs.pinot.apache.org/users/user-guide-query/multi-stage-query/explain-plan-1)
any of these two plans should tell us which predicate is the one that is not optimized to use indexes
a
Query1 plan with explainAskingServers=true
Copy code
Execution Plan
PinotLogicalExchange(distribution=[broadcast])
  LeafStageCombineOperator(table=[iot_advance_session_upsert])
    StreamingInstanceResponse
      StreamingCombineSelect
        SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[23811588])
          Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
            DocIdSet(maxDocs=[140000])
              FilterEmpty
        SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[37506293])
          Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
            DocIdSet(maxDocs=[220000])
              FilterAnd
                FilterBitmap
                FilterAnd
                  FilterInvertedIndex(predicate=[customer_id = '22370'], indexLookUp=[inverted_index], operator=[EQ])
                  FilterInvertedIndex(predicate=[msisdn IN (1000 ...msisdn LIST...)], indexLookUp=[inverted_index], operator=[IN])
                  FilterFullScan(predicate=[created_on >= '1761555455000'], operator=[RANGE])
query2 plan
Copy code
Execution Plan
LogicalUnion(all=[true])
  PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
    LogicalUnion(all=[true])
      PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
        LeafStageCombineOperator(table=[iot_advance_session_upsert])
          StreamingInstanceResponse
            StreamingCombineSelect
              SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[10816154])
                Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
                  DocIdSet(maxDocs=[60000])
                    FilterAnd
                      FilterBitmap
                      FilterAnd
                        FilterInvertedIndex(predicate=[customer_id = '22370'], indexLookUp=[inverted_index], operator=[EQ])
                        FilterInvertedIndex(predicate=[msisdn IN (350 ...msisdn LIST...)
                        FilterFullScan(predicate=[created_on >= '1761382655000'], operator=[RANGE])
              SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[54346245])
                Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
                  DocIdSet(maxDocs=[300000])
                    FilterEmpty
      PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
        LeafStageCombineOperator(table=[iot_advance_session_upsert])
          StreamingInstanceResponse
            StreamingCombineSelect
              SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[16299603])
                Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
                  DocIdSet(maxDocs=[90000])
                    FilterAnd
                      FilterBitmap
                      FilterAnd
                        FilterInvertedIndex(predicate=[customer_id = '22370'], indexLookUp=[inverted_index], operator=[EQ])
                        FilterInvertedIndex(predicate=[msisdn IN (350 ...msisdn LIST...)], indexLookUp=[inverted_index], operator=[IN])
                        FilterFullScan(predicate=[created_on >= '1761382655000'], operator=[RANGE])
              SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[48846471])
                Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
                  DocIdSet(maxDocs=[270000])
                    FilterEmpty
  PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]])
    LeafStageCombineOperator(table=[iot_advance_session_upsert])
      StreamingInstanceResponse
        StreamingCombineSelect
          SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[18156039])
            Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
              DocIdSet(maxDocs=[100000])
                FilterAnd
                  FilterBitmap
                  FilterAnd
                    FilterInvertedIndex(predicate=[customer_id = '22370'], indexLookUp=[inverted_index], operator=[EQ])
                    FilterInvertedIndex(predicate=[msisdn IN (300 ...msisdn LIST...)], indexLookUp=[inverted_index], operator=[IN])
                    FilterFullScan(predicate=[created_on >= '1761382655000'], operator=[RANGE])
          SelectStreaming(table=[iot_advance_session_upsert], totalDocs=[47006043])
            Project(columns=[[sgsn_ip, updated_on, apn_name, ggsn_ip, in_pkt_old, __metadata$offset, event_time_new, is_deleted, in_pkt_new, status_old, __key, event_time_old, nas_id, circle_name, msisdn, cell_id, status_new, out_pkt_new, session_id, start_time_new, version, dummy_start_rec, device_ip, end_time_new, rat_type, created_on, start_time_old, __metadata$partition, __metadata$recordTimestamp, ggsn_circle, imei, end_time_old, customer_id, disconn_reason, lac_id, out_pkt_old]])
              DocIdSet(maxDocs=[260000])
                FilterEmpty
my indexes for table
Copy code
"invertedIndexColumns": [
        "customer_id",
        "msisdn"
      ],
      "enableDefaultStarTree": false,
      "nullHandlingEnabled": false,
      "rangeIndexColumns": [
        "created_on"
      ],
y
What Pinot version are you running?
If you're on a version older than 1.4.0, then this could be caused by https://github.com/apache/pinot/issues/13617.
g
that's true but... that should not affect stagestats
y
It doesn't seem to right? The reported query latency is 120s which is not being reflected in the stage stats
g
Maybe that difference in stats was a spurious
Yep, makes sense
And given the IN perf issue was polynomial, a single IN with 1000 entries will be slower than 3 with 333
y
Yep exactly
Also the stats don't have 
clockTime
that you added recently, so the actual wall clock time would be 
executionTimeMs
/ 
parallelism
- somewhere around 1-2s for the first query? Which means that all the additional latency is coming from the broker side query compilation issue
d
a bit late to join this thread, but i have some question @Gonzalo Ortiz, i want to ask can we use 
numEntriesScannedPostFilter
result as the main indicator for evaluating the query performance? for context my 3 months report query gave this stage stage-- "numDocsScanned": 172002137, "totalDocs": 4420220697, "numEntriesScannedInFilter": 0, "numEntriesScannedPostFilter": 688008548, "numServersQueried": 2, "numServersResponded": 2, "numSegmentsQueried": 303, "numSegmentsProcessed": 303, "numSegmentsMatched": 303, "numConsumingSegmentsQueried": 0, "numConsumingSegmentsProcessed": 0, "numConsumingSegmentsMatched": 0, "minConsumingFreshnessTimeMs": 0, "numSegmentsPrunedByBroker": 2714, "numSegmentsPrunedByServer": 0, "numSegmentsPrunedInvalid": 0, "numSegmentsPrunedByLimit": 0, "numSegmentsPrunedByValue": 0, "brokerReduceTimeMs": 0, query is ran in 37 sec which is very poor i know we should have used star tree here but i am asking generally.
g
Not the main, but is important, yes
Open in Slack
PreviousNext
Powered by