@Rocky can you explain how bucketing works in a shared-data mode with 1 compute node or 3 compute nodes cluster

In StarRocks' shared-data mode (storage-compute separation), bucketing functions as the logical unit of data distribution and parallel processing. Unlike the shared-nothing architecture where data is physically tied to specific nodes, in shared-data mode, the data resides in central object storage (S3, GCS, Azure Blob, etc.), and Compute Nodes (CNs) "attach" to these buckets as needed. Here is how it works with 1 node versus 3 nodes: 1. The Core Principle: Tablets and Mapping When you define

DISTRIBUTED BY HASH(col) BUCKETS N

, StarRocks divides your table into N tablets. * Storage: These tablets are stored as files in object storage. * Management: The Frontend (FE) maintains a mapping of which CN is responsible for which tablet. * Replication: There is only one logical copy of the data in object storage (the storage layer handles its own redundancy), so you don't need to manage

replication_num

. ────────── 2. Scenario: 1 Compute Node Cluster * Assignment: The FE maps all N buckets to this single CN. * Processing: For any query, this single node is responsible for reading and processing all relevant buckets. * Parallelism: The parallelism is limited by the number of CPU cores on that single node. If you have 24 buckets and 16 cores, the node will process them in chunks. * Caching: This node will attempt to cache all hot data from all buckets on its local disk. 3. Scenario: 3 Compute Nodes Cluster * Distribution: The FE distributes the N buckets across the 3 nodes. For example, if you have 30 buckets, each CN will be assigned approximately 10 buckets. * Processing: When a query is executed, the FE's scheduler sends tasks to all 3 nodes. Each node only processes the buckets it "owns" in the mapping. * Parallelism: You now have the combined CPU/RAM power of 3 nodes. The query is processed in parallel across the cluster, significantly reducing latency for large datasets. * Caching: Each CN only caches the data for the specific buckets it is responsible for. This effectively triples your total available cache capacity compared to a single-node setup. ────────── 4. What happens during Scaling (1 $\to$ 3 nodes)? This is where the shared-data mode shines compared to shared-nothing: 1. No Data Migration: When you add 2 new CNs, StarRocks does not need to move large data files between nodes. 2. Metadata Update: The FE simply updates its internal mapping (e.g., "Buckets 11-30 are now handled by CN2 and CN3 instead of CN1"). 3. Cold Start/Cache Warm-up: The new nodes will initially experience "cache misses" because their local disks are empty. They will pull the data they need from the shared object storage. Once cached, they will perform at full speed. 4. Seamless Rebalancing: The rebalancing is near-instantaneous at the control plane level because only the responsibility for the data changes, not the data's physical location. Summary Recommendations * Bucket Count is Key: You should set the number of buckets based on your expected peak cluster size, not just your current 1 or 3 nodes. A common rule of thumb is to have enough buckets so that each CN handles multiple buckets (e.g., 8–16 buckets per node) to ensure high parallelism. * Minimum Buckets: Even on a 1-node cluster, don't use 1 bucket. If you later scale to 3 nodes, a 1-bucket table can only ever be processed by 1 node at a time, wasting the other 2 nodes's compute resources. Aim for a bucket size of roughly 1GB to 5GB for optimal performance. References • faq/shared_data_faq.md • administration/management/resource_management/Replica.md • faq/Deploy_faq.md • table_design/table_types/primary_key_table.md