@Rocky What is the maximum cluster size? I'd like to know the number of FEs and BEs. The reasoning is that I'm unsure whether it's better to scale up each node to increase the number of CPU cores and memory modules, or to add nodes to the cluster, in order to ensure scalability. If scaling out is the case, I believe the advantage is the ability to add capacity in finer increments, such as 4vcpu or 8vcpu.

There is no strict hard limit on the maximum number of nodes in a StarRocks cluster, but there are practical recommendations and architectural trade-offs to consider for both Frontend (FE) and Backend (BE) nodes. 1. Frontend (FE) Nodes FE nodes manage metadata and query planning. * Follower FEs: In production, it is recommended to deploy 3 Follower nodes (1 Leader + 2 Followers) to ensure High Availability (HA). While you can have 5, the Raft protocol used for metadata consistency incurs higher latency with more voting members. * Observer FEs: If you need to handle higher query concurrency, you can add Observer nodes. There is no hard limit on the number of Observers, but practically they are limited by the Leader's capacity to synchronize the edit log to them. * Scalability: FEs do not typically require massive horizontal scaling because they only handle planning, not data processing. 2. Backend (BE) / Compute Nodes (CN) BE nodes handle data storage and execution. * Recommended Size: At least 3 nodes are required for data redundancy (default 3 replicas). * Maximum Size: There is no fixed software limit. Large-scale production clusters in the community often exceed 100+ nodes. * Constraint: The real limit for BE scaling is the total number of tablets in the cluster. Each FE must keep metadata for all tablets in memory. A standard FE with 64GB of RAM can typically manage around 1 million tablets. If your cluster grows so large that the tablet count exceeds FE memory, you would need to scale up the FE RAM or optimize your partitioning/bucketing to reduce tablet count. 3. Scaling Up vs. Scaling Out Your reasoning about scaling out (horizontal) is generally aligned with StarRocks' design philosophy: * Scaling Out (Horizontal - Adding Nodes): * Advantage: Better for high concurrency. More nodes mean more independent workers to handle simultaneous requests. * Incremental Growth: As you noted, you can add nodes in smaller increments (e.g., 8 vCPU nodes) to match your growth closely. * Resiliency: More nodes reduce the impact of a single node failure. * Scaling Up (Vertical - Larger Nodes): * Advantage: Better for complex, large-scale single queries. If a single query needs to process a massive amount of data in memory (e.g., a huge join), a single node with 256GB RAM will perform better than many small nodes because it avoids network shuffle overhead. * Recommendation: For production BEs, the "sweet spot" is often nodes with 16–32 vCPUs and 64–128GB RAM. Summary Recommendation: If your workload is primarily about handling many users (concurrency), scale out. If you are struggling with single, massive queries hitting memory limits, scale up each node to at least 64GB or 128GB before continuing to scale out. References • faq/Deploy_faq.md • administration/management/Scale_up_down.md • deployment/plan_cluster.md • administration/management/FE_configuration.md • introduction/Architecture.md

@Rocky Also, is there a recommendation for which is better for managing a cluster: EKS + Fargate or EC2 + Auto Scaling Groups? I think EC2 is superior because it allows for easier performance tuning using bare metal. On the other hand, if the number of nodes in the cluster increases significantly and fluctuates constantly, EKS seems more suitable.

Between EKS + Fargate and EC2 + Auto Scaling Groups (ASG), your intuition is largely correct: EC2 (specifically via EKS Managed Node Groups) is significantly superior for StarRocks. While EKS is excellent for management, Fargate is generally not recommended for a high-performance OLAP database like StarRocks for several technical reasons. 1. Why Fargate is often avoided for StarRocks * Lack of AVX2 Support: StarRocks relies heavily on the AVX2 instruction set for its vectorized execution engine. Fargate abstracts the hardware to the point where you cannot guarantee or optimize for these specific CPU instructions, leading to a massive performance penalty. * System Tuning Limits: StarRocks requires specific OS-level optimizations for high performance, such as modifying

ulimits

, enabling

huge pages

, and adjusting kernel network parameters. Fargate restricts access to these settings. * Storage Performance: Backend (BE) nodes require high-throughput, low-latency disk access (ideally local NVMe SSDs). Fargate's ephemeral storage and EBS integration are typically slower and less configurable than what you can achieve with EC2 instance types like the I4i family. 2. EKS + EC2 (Recommended Middle Ground) The most common production setup in the community is EKS using EC2 Managed Node Groups. This gives you the best of both worlds: * Management: You use the StarRocks Kubernetes Operator, which handles the lifecycle of the cluster (deploying FEs and BEs, managing upgrades). * Performance: You can select specific EC2 instance types (like

c6i

or

i4i

) that support AVX2 and local SSDs. * Automation: The Operator makes scaling easier than manual EC2 management. For example, when you scale out the BE count in your YAML, the Operator handles the registration of new nodes automatically. 3. Scaling Behavior: BE vs. CN When considering scaling out in "finer increments," it is important to distinguish between node types: * Backend Nodes (BE): These store data. Scaling out is easy, but scaling in is slow because data must be "decommissioned" (rebalanced to other nodes) to prevent data loss. * Compute Nodes (CN): In StarRocks' Shared-Data mode, you can use stateless Compute Nodes. These are perfect for the "EKS + Auto Scaling" approach. You can use the Horizontal Pod Autoscaler (HPA) to add/remove CNs based on CPU/RAM usage. Since they don't store local data, they can scale up and down in minutes. Recommendation * For Production: Use EKS with EC2 Managed Node Groups. It allows the StarRocks Operator to handle the "constant fluctuation" you mentioned while keeping the performance tuning benefits of EC2. * Avoid Fargate: Unless you are running a very small, non-performance-critical development instance. * Scaling Strategy: Use a baseline of BE nodes for your core data storage and use CN nodes for the "fine increment" scaling to handle spikes in query demand. References • deployment/sr_operator.md • deployment/deployment_prerequisites.md • administration/management/Scale_up_down.md • deployment/shared_data/feature-support-shared-data.md References • deployment/sr_operator.md • administration/management/Scale_up_down.md • administration/stargo.md