@Rocky If I need to split a large shared-data cluster into multiple smaller clusters by domain, is there a way to do this without re-ingesting data from upstream sources and does the fact that data already lives in S3 enable any zero-copy or low-cost migration path?

Splitting a large shared-data (Cloud Native) cluster into smaller clusters by domain can be achieved effectively through two primary paths, depending on whether you need logical isolation (separate compute) or physical isolation (separate control planes). 1. Zero-Copy: Multiple Compute Clusters (Warehouses) The most efficient and "zero-copy" way to split your cluster by domain is to use the Multi-Warehouse (Compute Clusters) feature. * How it works: In a shared-data architecture, data is already decoupled from compute. You can create multiple separate "Compute Clusters" (CN node groups) within the same StarRocks instance. Each domain (e.g., Finance, Marketing) can be assigned its own compute cluster. * Benefits: * Zero-Copy: All compute clusters point to the same data in S3. There is no data migration or re-ingestion required. * Resource Isolation: Queries from one domain won't consume CPU/Memory from another. * Low Cost: You only scale compute nodes for each domain as needed, while storage remains shared and managed centrally. * Constraint: They still share the same Frontend (FE) control plane and metadata. 2. Low-Cost Migration: Cross-Cluster Data Migration Tool If you require separate physical clusters (distinct FE control planes), you cannot simply "attach" the existing S3 data to a new FE because metadata (tablet IDs, versions) is tightly coupled with the FE. However, you can use the community-provided StarRocks Cross-cluster Data Migration Tool (

starrocks-cluster-sync

). * Mechanism: This tool automates the migration from a source cluster to a target cluster. While it is often used for shared-nothing to shared-data migrations, it also supports shared-data to shared-data migration. * Migration Path: It replicates the data from the source cluster to the target. While this involves writing new data to the target's S3 bucket (so it isn't strictly zero-copy of the underlying files), it is far lower cost and effort than re-ingesting from upstream sources (like Kafka or Spark), as it maintains the schema and relational integrity. * Documentation: You can find this tool in the official Cross-cluster Data Migration Tool guide. 3. Cluster Snapshot (Disaster Recovery - v3.4.2+) Starting from StarRocks v3.4.2, the Cluster Snapshot feature (currently in Beta) allows you to take snapshots of metadata and data on shared-data clusters. * While primarily designed for disaster recovery, this can technically be used to recover a "clone" of the cluster into a new environment. _ This is the closest path to a "metadata-level migration" for shared-data clusters, but it currently recovers the _entire* cluster rather than specific domains. Summary Table Path | Isolation Level | Migration Cost | Zero-Copy? Compute Clusters | Logical (Compute only) | None | Yes Migration Tool | Physical (Full Cluster) | Low (Automated) | No (Data is replicated) Cluster Snapshot | Physical (Full Cluster) | Low (Beta) | Partial (Metadata + Data) Recommendation: If your goal is simply to ensure domain-specific workloads don't interfere with each other, use Compute Clusters. If you must have entirely separate FE management, use the Cross-cluster Data Migration Tool. Note: As of version 3.4, standard

BACKUP

and

RESTORE

commands are not yet supported for shared-data (Cloud Native) tables. References • quick_start/shared-data.md • deployment/shared_data/feature-support-shared-data.md • knowledge/admin_and_operation/shared_data_compaction.md