hey team. wondering how is pinot upsert realtime t...
# generalc
Chengxuan Wang
07/02/2024, 10:18 PMhey team. wondering how is pinot upsert realtime table get partitioned into segments? is it based on the time col or the kafka message key? thanks
Chengxuan Wang
07/02/2024, 10:19 PM@Jackie
j
Jackie
07/02/2024, 10:24 PMTake a look at this doc and see if it answers your question
Jackie
07/02/2024, 10:24 PMEssentially primary key should already be partitioned on the Kafka side
Jackie
07/02/2024, 10:24 PMThere is no extra partition on pinot side
c
Chengxuan Wang
07/02/2024, 10:28 PMah. got it. thanks.
Chengxuan Wang
07/05/2024, 4:23 PMFollow up question: given the messages with same key will always go to same server, wondering if there will be hot spot issue? Like some keys are frequently updated.
j
Jackie
07/08/2024, 6:02 PMPossible if the data distribution is very skewed. In normal case, the cardinality of the primary key is usually high. Given much fewer partitions than the keys, the data within each partition is more or less evenly distributed