https://pinot.apache.org/ logo
#general
Title
# general
d

Dan DC

09/28/2021, 10:28 AM
I've also got a question about partitioning in hybrid tables. If I understand correctly this only applies to offline tables. Does
segmentPartitionConfig
play together with the time column? The field only accepts 1 value at the moment and I was wondering whether segments are generated using
timeColumnName
and further partitioned using the
segmentPartitionConfig
? If I don't specify partitioning then the segments are effectively replicated to all the servers?
m

Mayank

09/28/2021, 1:49 PM
This config helps partition data on a primary key. You typically don’t want to choose time column for this, but a column that appears on most queries with equality predicate. If you don’t use any partitioning or replica group based assignment, then each segment can go to any of the servers (n copies).
d

Dan DC

09/28/2021, 2:51 PM
Thank you Mayank. So the attribute
segmentConfig.timeColumnName
is not used in an offline table?
I guess if I need to partition my data, let's say, by hour then I need to inject a new field in my data and use it the partition config of the offline table
m

Mayank

09/28/2021, 4:16 PM
No no, I am saying time column already gets special treatment in terms of time based pruning already. If you partition your data on time column, you don't need to explicitly tell Pinot. The
segmentPartitionConfig
is more for a primary key based partitioning.
d

Dan DC

09/28/2021, 4:37 PM
Got it, thanks for clarifying that