https://pinot.apache.org/ logo
#general
Title
# general
h

Harish Bohara

04/14/2022, 10:50 PM
Hi.. if I have a large number of segments (for realtime tables). Is there a setting which merges segment at background? Or Any cron Job?
m

Mayank

04/14/2022, 10:54 PM
Yep minion tasks
h

Harish Bohara

04/14/2022, 10:55 PM
It seems it is only for offline tables: https://docs.pinot.apache.org/operators/operating-pinot/minion-merge-rollup-task The Minion merge/rollup task allows a user to merge small segments into larger ones, through which Pinot can potentially benefit from improved disk storage and the query performance. For complete motivation and reasoning, please refer to the design doc above. Currently, we only support OFFLINE table APPEND use cases.
You can use this in conjunction with managed offline flow
h

Harish Bohara

04/14/2022, 10:58 PM
👍