@Rocky what is the recommendation for geo-distributing StarRocks clusters? Do we stretch a single cluster across multiple zones/data centers? or do we create multiple clusters and link them together with external catalogs (we tried this and it's slow)? How does StarRocks handle query fragmentation in a geo-distributed deployment? How does it handle consistency/replication? Are these features supported at all natively?

For geo-distributed StarRocks deployments, the recommendation depends heavily on the network latency between your locations. 1. Multi-Zone (Same Region) If you are deploying across multiple Availability Zones (AZs) in the same region (sub-10ms latency), the standard recommendation is to stretch a single cluster across these zones. * How it works: You use Rack-Awareness (or label-aware location) to tell StarRocks which Backend (BE) nodes belong to which zone. * Replication: When you create a table with

replication_num = 3

, StarRocks will automatically ensure that replicas of the same tablet are stored in different zones. * Consistency: Data is written synchronously to a majority of replicas. This provides high availability (HA) and strong consistency; if one zone goes down, the cluster remains operational. * Query Performance: While there is a slight latency penalty for cross-zone shuffles, the StarRocks MPP engine is designed to handle this within a region. 2. Multi-Region (Geo-Distribution) If your data centers are geographically distant (e.g., US-East vs. Europe), you should NOT stretch a single cluster. The high latency (>50-100ms) will severely degrade write performance (waiting for majority ACKs) and query performance (shuffling data across the WAN). The recommendation for multi-region is to create multiple clusters and link them using Cross-Cluster Replication (CCR). * CCR (Native Feature): StarRocks supports a dedicated Cross-Cluster Replication service. It replicates data asynchronously from a primary cluster to a standby cluster at the database or table level. * Consistency: This is asynchronous replication. The standby cluster is typically read-only and serves local queries with low latency, while writes happen at the primary. * Why not External Catalogs? You found this slow because an external catalog query is a federated query—data is pulled over the network at query time. CCR, by contrast, replicates the data ahead of time so queries run against local disks. 3. Query Fragmentation and Geo-Distribution StarRocks does not currently have "geography-aware" query fragment scheduling for a single cluster. * In a Stretched Cluster: The FE (Frontend) schedules "Query Fragments" (MPP execution units) across all available BEs based on data locality. If a query requires a shuffle (e.g., a Join or Group By), data will be transferred across regions if the cluster is stretched. Because StarRocks assumes a high-speed local network, it will not attempt to minimize cross-region traffic, leading to "fragmentation" that is bottlenecked by WAN speeds. * In Multi-Cluster (CCR): Queries are entirely local to each cluster, so there is no cross-region fragmentation. Summary of Support Feature | Native Support | Recommendation Rack/Zone Awareness | Yes (

label_aware_location

) | Use for multi-AZ (same region). Cross-Cluster Replication | Yes (CCR Tool/Service) | Use for multi-region (DR/Geo-locality). Stretched Cluster | Technical possible | Only if latency is <10ms. Consistency | Majority Quorum (Stretched) | Async (Cross-Cluster). Next Steps: If you are doing multi-region, look into the StarRocks CCR tool. If you are doing multi-zone, configure your BEs with labels in

be.conf

(e.g.,

broker_client_timeout_ms

is not it, you use the

ALTER SYSTEM ADD BACKEND

with labels or set them in the configuration) and ensure

enable_pipeline_engine

is active for best performance. References • integrations/loading_tools/SMT.md • administration/data_migration_tool.md • faq/operation_maintenance_faq.md • administration/management/resource_management/Replica.md