How can I get star tree like performance in Hybrid...
# troubleshooting
r
How can I get star tree like performance in Hybrid Table for the Upsert realtime table?
x
upsert table doesn’t support startTree index right now
r
yeah but when applying star tree on an Hybrid table and then querying on those star tree fields will give result with star tree using if it exists in the Offline Segments but when done on Hybrid Segments it takes much time and if we want to improve the performance of Hybrid table(Realtime without star tree) what other indices are the best to use?
x
your realtime table is append table or upsert table ?
if append table, you could also build startree index
r
no its upsert table
that's why facing this issue
x
i see. whats the query and primary key in your case
r
so, I have setup this
Copy code
"starTreeIndexConfigs": [
    {
      "enableStarTreeIndex": true,
      "dimensionsSplitOrder": [
        "o_shipping_method",
        "o_payment_method",
        "s_courier",
        "s_status",
        "s_company_id"
      ],
      "skipStarNodeCreationForDimensions": [],
      "functionColumnPairs": [
        "SUM__o_net_total_double",
        "SUM__o_total_double",
        "SUM__a_applied_weight_amount_double",
        "SUM__a_charge_weight_amount_double",
        "COUNT__*"
      ],
      "maxLeafRecords": 10000
    }
  ],
in offline table and I am facing performance issue in the case when the query is running on Hybrid segments. when querying on separate offline or Online segments it was running under 100 ms and with hybrid table it was taking 900-1000 ms. the query:
Copy code
SELECT o_shipping_method,
  o_payment_method,
  s_courier,
  s_status,
  s_company_id,
  COUNT(*) AS shipment_count
FROM shipmentMerged_v3
GROUP BY 1,2,3,4,5
and primary key = s_id
@Xiang Fu any suggestion?
x
i see, in this case the upsert table needs to do the runtime scanning
startree index on upsert table is ongoing but not yet there
r
Oh, I mean that's still great news. Waiting for it. Meanwhile we'll use other pinot features. 👍🏽
x
👍