This message was deleted.
# generals
Slackbot
09/06/2023, 3:11 AMThis message was deleted.
l
Laksh Singla
09/06/2023, 4:05 AMIt did: https://github.com/apache/druid/pull/14310, however this should have strictly improved on the partitions that you get instead of reducing it
Laksh Singla
09/06/2023, 4:07 AMWhat are the segment sizes for a particular time segment? MSQ has num rows per segment set to 3mn. Are there segments that are super larger than this?
k
Karan Kumar
09/06/2023, 4:40 AM@Arav K If you could send the segment ID -> numRows, size between the 2 versions it will help us looks at the root cause.
You can always use the undocumented parameter
Copy code
clusterStatisticsMergeMode=PARALLELKaran Kumar
09/08/2023, 1:49 AM@Arav K Any updates on this ?
➕ 1
a
Arav K
09/08/2023, 5:25 AMFor new version . Below is the requested metrics . (partitioned by Day , rowsPerSegment 7.5M)
Arav K
09/08/2023, 5:34 AMFor the old version . I have statistics for same dataset , but only for slightly different partitioning configuration - Partitioned by ALL , , rowsPerSegment 7.5M @Karan Kumar @Laksh Singla
k
Karan Kumar
09/10/2023, 12:39 PMCan you also share the segment size along with numRows. MSQ uses data sketch for determining partition boundaries. Partitioned by all forces all partition to a single time chunk which should ideally be worse since you would be affected by data skech downsizing the most. More the data sketch is down sampled, less accurate are the partition boundaries.