<@U0A71G31CDV> how can I get the table from shared...
# questions-and-troubleshooting
m
@Rocky how can I get the table from shared nothing cluster to shared data cluster? is it possible in any way one time?
r
Moving a table from a shared-nothing (classic) cluster to a shared-data (cloud-native) cluster cannot be done via an in-place transformation or `BACKUP`/`RESTORE` (as shared-data clusters do not yet support the standard Backup/Restore mechanism). Instead, you can perform a one-time migration using one of the following methods: Method 1: Using the StarRocks Catalog (Recommended) This is the most efficient way to move data directly between clusters. You create a "catalog" in your new shared-data cluster that points to your old shared-nothing cluster. 1. In the Shared-Data Cluster: Create a StarRocks catalog to link to the source cluster.
Copy code
sql
    CREATE EXTERNAL CATALOG source_cluster
    PROPERTIES (
        "type" = "starrocks",
        "starrocks.host" = "<source_fe_ip>",
        "starrocks.port" = "9020", -- RPC port
        "starrocks.user" = "root",
        "starrocks.password" = "password"
    );
2. Create the target table: Create the table schema in the shared-data cluster. Ensure you include shared-data properties like
storage_volume
. 3. Run the migration: Use an
INSERT INTO ... SELECT
statement.
Copy code
sql
    INSERT INTO target_db.target_table
    SELECT * FROM source_cluster.source_db.source_table;
Method 2: Export and Import via Object Storage If the clusters cannot communicate directly over the network, use intermediate object storage (S3, GCS, etc.). 1. Export from Shared-Nothing:
Copy code
sql
    -- In the source cluster
    EXPORT TABLE source_table
    TO "<s3://my-bucket/migration_data/>"
    PROPERTIES (
        "column_separator" = ",",
        "format" = "parquet"
    );
2. Import to Shared-Data: Use the
FILES()
function to pull the data into your new table.
Copy code
sql
    -- In the target cluster
    INSERT INTO target_table
    SELECT * FROM FILES(
        "path" = "<s3://my-bucket/migration_data/*>",
        "format" = "parquet"
    );
Important Notes * No Transformation: You cannot "convert" an existing cluster from shared-nothing to shared-data mode; you must set up a new shared-data cluster and move the data. * Table Type: Tables in a shared-data cluster are of type
CLOUD_NATIVE
. While they support the same models (Duplicate, Primary Key, etc.), the underlying storage mechanism is different. * Metadata: You must manually recreate the table schemas (
CREATE TABLE
) in the target cluster before loading the data. References • deployment/shared_data/feature-support-shared-data.md • deployment/shared_data/minio.md • deployment/shared_data/s3.md • deployment/shared_data/azure.md • administration/management/Backup_and_restore.md • deployment/shared_data/gcs.md
m
@Rocky looks like there is not starrocks type external catalog. I couldn't find in documentation as well , getting this ERROR 1064 (HY000): Getting analyzing error. Detail message: [type : starrocks] is not supported.
r
I apologize for the confusion in my previous response. You are correct—*there is no
starrocks
type for external catalogs.* The
CREATE EXTERNAL CATALOG
statement does not currently support a native StarRocks connector, which is why you encountered the
[type : starrocks] is not supported
error. For a one-time migration from a shared-nothing cluster to a shared-data cluster, you should use one of the following official methods: Method 1: StarRocks Cross-cluster Data Migration Tool (Recommended) This is the official tool (also known as
starrocks-cluster-sync
) designed for this exact scenario. It supports migrating data from shared-nothing to shared-data clusters. 1. Download the tool:
Copy code
bash
    wget <https://releases.starrocks.io/starrocks/starrocks-cluster-sync.tar.gz>
    tar -xvzf starrocks-cluster-sync.tar.gz
2. Configure for one-time migration: Edit
conf/sync.properties
and set: *
one_time_run_mode=true
(This ensures it runs once and exits). *
source_fe_host
and
target_fe_host
details. * `target_cluster_storage_volume`: Specify your shared-data storage volume name. 3. Run the tool: It will automatically handle the schema recreation and data transfer. Method 2: Using the JDBC Catalog (The "Pull" Method) Since StarRocks is MySQL-compatible, you can use a JDBC Catalog with the MySQL driver to "see" your shared-nothing cluster from your shared-data cluster. 1. In the Shared-Data Cluster:
Copy code
sql
    CREATE EXTERNAL CATALOG source_sn_cluster
    PROPERTIES (
        "type" = "jdbc",
        "user" = "root",
        "password" = "your_password",
        "jdbc_uri" = "jdbc:mysql://<source_fe_ip>:9030/source_db",
        "driver_url" = "<https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.28/mysql-connector-java-8.0.28.jar>",
        "driver_class" = "com.mysql.cj.jdbc.Driver"
    );
2. Run the migration:
Copy code
sql
    INSERT INTO target_table SELECT * FROM source_sn_cluster.source_db.source_table;
Method 3: Export/Import via Object Storage (The "Manual" Method) If the two clusters cannot communicate directly, you can use intermediate storage: 1. Export from the shared-nothing cluster to S3/MinIO/GCS using the
EXPORT
command. 2. Import into the shared-data cluster using the
INSERT INTO ... SELECT * FROM FILES(...)
command. Summary of Compatibility * Version Note: For the Migration Tool to work with shared-data, your target cluster should be version v3.1.8, v3.2.3, or later. * Backup/Restore: Traditional
BACKUP
and
RESTORE
commands are not currently supported for shared-data clusters. References • administration/data_migration_tool.md • sql-reference/sql-statements/Catalog/CREATE_EXTERNAL_CATALOG.md • data_source/catalog/jdbc_catalog.md References • administration/data_migration_tool.md