@Rocky when we set a table `DISTRIBUTED BY HASH(

id

)` , what's the default value picked for number of buckets? Is it some global default? Also, can starrocks automatically change/alter the number of buckets in a table over time?

Since StarRocks v2.5.7, you do not need to specify the number of buckets manually. If you omit the

BUCKETS

keyword, StarRocks uses Automatic Bucket Distribution. 1. Default Number of Buckets The initial number of buckets is determined by a global FE configuration

enable_auto_tablet_distribution

(default is

true

). If you have a new table with no data, StarRocks calculates the initial bucket count based on the number of active Backend (BE) nodes (X) using the following formula: * X <= 12:

2 * X

* X <= 24:

1.5 * X

* X <= 36:

36

* X > 36:

min(X, 48)

2. Can StarRocks change the number of buckets over time? The behavior depends on whether you are talking about existing data or new partitions: * For New Partitions (Automatic): Yes. For partitioned tables, StarRocks automatically adjusts the bucket count for newly created partitions based on the data volume of the previous 5 partitions. This ensures that as your data grows, new partitions will have an optimal number of buckets. * For Existing Partitions (Manual/Dynamic): * Hash Bucketing: The number of buckets in an existing partition is fixed once created. However, since v3.2, you can manually alter them using:

sql
        ALTER TABLE <table_name> DISTRIBUTED BY HASH(<column>) BUCKETS <new_number>;

This triggers a background data redistribution (similar to a schema change). * Random Bucketing (Dynamic): Since v3.2, if you use

DISTRIBUTED BY RANDOM

and set the

bucket_size

property (e.g.,

PROPERTIES("bucket_size" = "1073741824")

), StarRocks can automatically and dynamically increase the number of buckets within a partition during data loading as the volume grows. Summary * Initial count: Based on the number of BEs. * Evolution: For Hash tables, StarRocks automatically tunes the bucket count for future partitions but requires a manual

ALTER

to change existing ones. Only Random bucketing supports fully dynamic "on-the-fly" bucket increases within a single partition. References • knowledge/usage/best-practice.md • table_design/data_distribution/Data_distribution.md • administration/management/FE_configuration.md

And what if this isn't enabled?

enable_auto_tablet_distribution

What bucket value is set? @Rocky

If

enable_auto_tablet_distribution

is disabled (set to

FALSE

), there is no hidden global default value (like a hardcoded "10"). Instead, the following rules apply: 1. Manual Specification is Required If you omit the

BUCKETS

keyword while

enable_auto_tablet_distribution

is

FALSE

, the

CREATE TABLE

statement will typically fail with a syntax or semantic error. You must manually specify the number of buckets at table creation, for example:

sql
DISTRIBUTED BY HASH(id) BUCKETS 10;

2. Partition Inheritance If the table is already created with a specific bucket count (e.g., 10), and you later add a new partition without specifying the bucket count, the new partition will inherit the bucket count defined at the table level. 3. If

enable_auto_tablet_distribution

is ENABLED (Default) When enabled (the default since v2.5.7), the "default" isn't a fixed number but a formula based on your cluster's Backend (BE) count (X): * X <= 12:

2 * X

* X <= 24:

1.5 * X

* X <= 36:

36

* X > 36:

min(X, 48)

4. Automatic Changes Over Time StarRocks handles "evolution" of bucket counts differently depending on the method: * For Hash Bucketing (Standard): * New Partitions: StarRocks automatically tunes the bucket count for future partitions based on the data volume of the last 5 partitions. * Existing Partitions: The bucket count is fixed once a partition is created. It does not automatically "re-bucket" existing data. However, since v3.2, you can manually trigger a change using:

sql
        ALTER TABLE <table_name> DISTRIBUTED BY HASH(id) BUCKETS <new_number>;

* For Random Bucketing (Dynamic): If you use

DISTRIBUTED BY RANDOM

and set a

bucket_size

property (e.g.,

PROPERTIES("bucket_size" = "1073741824")

), StarRocks can automatically and dynamically increase the number of buckets within an existing partition during data loading as it grows. Summary * Auto-enabled: Formula-based (depends on BE count). * Auto-disabled: You must specify it; there is no global fallback number. * Over time: Only new partitions or

RANDOM

bucketing with

bucket_size

change automatically;

HASH

bucketing on existing data requires a manual

ALTER

. References • table_design/data_distribution/Data_distribution.md • knowledge/usage/best-practice.md • administration/management/FE_configuration.md