Running Starrocks 4.1.3, Wanted to check if there is any plan to support this adaptive layout feature in shared-nothing clusters? We have Native BE based tables and we are seeing issues because tablet size have gone past 300GB.

Hello! I'm Rocky, your StarRocks assistant. How can I help you today? Please let me know your specific question or the issue you're facing.

@Rocky Running Starrocks 4.1.3, Wanted to check if there is any plan to support this adaptive layout feature in shared-nothing clusters? We have Native BE based tables and we are seeing issues because tablet size have gone past 300GB.

In StarRocks 4.1, the Adaptive Layout feature (which includes dynamic tablet splitting/merging and Range Distribution) is currently exclusive to shared-data (cloud-native) clusters. Support and Roadmap As of StarRocks 4.1.x, there is no official timeline for bringing the full Adaptive Layout suite to shared-nothing (Native BE) clusters. The primary reason for this phased rollout is architectural: * Shared-Data: Tablets are stored in object storage. Splitting or merging tablets is primarily a metadata operation that doesn't require moving large amounts of physical data between nodes. * Shared-Nothing: Tablets are tied to local disks on specific BEs. Dynamically splitting or merging them requires physical data shuffling and re-replication across the cluster to maintain balance and redundancy, which is significantly more complex and resource-intensive. Addressing your 300GB Tablet Issue A tablet size of 300GB is significantly beyond the recommended range for shared-nothing clusters. In this mode, StarRocks best practices recommend keeping tablets between 1GB and 10GB (compressed). Running with 300GB tablets in a Native BE environment typically leads to: * High Memory/CPU Pressure: Ingestion and compaction tasks for a single massive tablet cannot be parallelized across threads or nodes, creating a performance bottleneck. * I/O Amplification: Queries scanning even a small amount of data within that tablet may trigger large, inefficient I/O operations. * Recovery Overhead: If a BE fails, restoring a 300GB tablet takes much longer than restoring many smaller tablets, impacting cluster availability. Recommendations for Shared-Nothing Since you are on 4.1.3, you can use features introduced in recent versions to resolve this manually without a full table rebuild: 1. Modify Bucket Count: Since v3.2, you can modify the number of buckets for existing partitions or the whole table using

ALTER TABLE

. For example:

sql
    -- Modify the default bucket number for future partitions
    ALTER TABLE <table_name> MODIFY DISTRIBUTION DISTRIBUTED BY HASH(<cols>) BUCKETS <new_number>;

2. Adjust Existing Partitions: You can also adjust the bucket count of existing partitions to force a redistribution into smaller tablets:

sql
    ALTER TABLE <table_name> MODIFY PARTITION (<partition_name>) SET ("buckets" = "<new_number>");

3. Check for Skew: If one tablet is 300GB while others are small, check your

DISTRIBUTED BY

key. Ensure it has high enough cardinality to distribute data evenly. If your data growth is unpredictable and you cannot easily manage tablet sizes, you may want to evaluate migrating to a shared-data architecture (using S3/HDFS), where the 4.1 Adaptive Layout features can manage these transitions automatically. References • deployment/shared_data/feature-support-shared-data.md • quick_start/shared-data.md • deployment/shared_data/hdfs.md • deployment/shared_data/azure.md