Hi Everyone, Question about hashed partition in c...
# general
v
Hi Everyone, Question about hashed partition in compaction. I have datasource with multi tenant information with customer_id column. Currently we have enabled dynamic partitioning. However, if we enable customer_id as partitioning with hashed option, then the segments that are saved in s3 are bucketd with number of partitions as well. If we run MSQ with customer_id as filter, does it pull all the segment files across partition buckets or only the buckets which contain customer_id from S3 and process only this data in MSQ ?
@Will Xu @John Kowtko would you be able to provide info here. Thanks in advance.
w
I think MSQ partition elimination only works with multi-dim (i.e. range) partitioning, not with hash.
So of the 3 options: dynamic, hash, and multi-dim, the last one works.
v
thanks @Will Xu