How does segment size impact offline tables? Does the offline segment ingestion job always create one segment regardless of the number of records, or is it smart enough to create multiple segments of optimal segment size?
m
Mark Needham
03/03/2022, 3:19 PM
generally it creates one segment per source file from my understanding
m
Mayank
03/03/2022, 3:30 PM
As of now in the OSS yes 1 input file = 1 segment
b
Bobby Richard
03/03/2022, 4:08 PM
So input files should be sized to create the desired pinot segment size for optimal query performance?