Hi everyone, I have a question about random bucketing with duplicate tables. From what I see, when we don’t explicitly set the number of buckets, StarRocks automatically assigns it. In our case, for Duplicate tables, the bucket count is always set to 5. This happens even though we have more than 100 backends in the cluster. And it doesn’t seem to reflect the actual incoming traffic from our Kafka streaming routine load. Then, the load is not correctly distributed over the Starrocks cluster. - Is this the expected behavior for duplicate tables? - Why does the system default to 5 buckets in this case? - And is there a way to have StarRocks auto-adjust the bucket count for duplicate tables based on real data volume ? Thanks a lot!

Hello, when you don't mention the bucket size, it's determined based on the number of BE you have on your clusters. For the bucket_size properties (only works on random bucketing), the table will create more sub-partitions as more data comes in. Each sub-partition will have its own buckets - it will have more buckets as the partitions grow.

Thanks a lot for your answer! So just to confirm — if we don’t specify the

bucket_size

, does that mean the number of buckets will be the same for all tables, regardless of their size? In our case, we notice that both very small and very large tables end up with the same number of buckets, which isn’t ideal. As a result, only 5 backends are actually working at the same time, even though we have more than 100 available. Do you think it would make sense to open an issue on GitHub to suggest an improvement — for example, to make the bucket number depend on the incoming data volume? And one last question: why is the behavior different between Aggregate and Duplicate tables?

I think I was bit misleading. I was trying to say if you don't mention the bucket count (during table creation), the number of bucket is automatically chosen and is fixed, unless modified manually using

alter table

. The number of buckets can only grow automatically if your table is using random bucketing and

bucket_size

property is set. For the behavior question, aggregate tables do store copies of rows with the same keys, but instead merge them into a single row - no point of using random bucketing since all the rows with the same key has to live in single tablet.