Hi.. if I have a large number of segments (for rea...
# generalh
Harish Bohara
04/14/2022, 10:50 PMHi.. if I have a large number of segments (for realtime tables). Is there a setting which merges segment at background? Or Any cron Job?
m
Mayank
04/14/2022, 10:54 PM
Yep minion tasks
Mayank
04/14/2022, 10:55 PM
h
Harish Bohara
04/14/2022, 10:55 PMIt seems it is only for offline tables: https://docs.pinot.apache.org/operators/operating-pinot/minion-merge-rollup-task
The Minion merge/rollup task allows a user to merge small segments into larger ones, through which Pinot can potentially benefit from improved disk storage and the query performance. For complete motivation and reasoning, please refer to the design doc above. Currently, we only support OFFLINE table APPEND use cases.
m
Mayank
04/14/2022, 10:57 PM
Mayank
04/14/2022, 10:57 PM
You can use this in conjunction with managed offline flow
h
Harish Bohara
04/14/2022, 10:58 PM👍