This message was deleted.
# troubleshooting
s
This message was deleted.
s
@Didip Kerabat I think it is more a question of how big you want target the segments to be. It is not an exact row count cutoff as it is in dynamic partitioning. It will sort the partitioning dimension values and calculate a distribution of rows across the values for each time chunk, then use the target rows per segment to find range boundaries (dimension values) that create segments that are closest to the target row count.
d
If I mistakenly put the number too high, I guess the only downside is that the segment file size is too big?
s
Yeah, I guess. You'll also want to check the resulting segments, if your partitioning dimension values have significant skew you will see larger segments when they end up with those values. One of the advantages of multi-dim is that you can add another dimension to alleviate the skew since it will find combinations of values as the boundaries and be able to better split rows across them.
a
Also, the processing of a segment cannot be parallelized. If you have really big segments, you won't be benefitting from parallel processing. You can set it to the average number of rows in the chunk that is currently dynamically partitioned. target rows = (total number of rows) / (total number of segments) with that, you should get the same number of segments but evenly balanced and better sized.
❤️ 1
d
That seems easy to follow, thanks @Abhishek Agarwal