http://starrocks.io logo
Join Slack
Powered by
This message was deleted.
# questions-and-troubleshooting
s
This message was deleted.
r
Yes, it is possible to connect two StarRocks instances via a JDBC connection, but the error you are seeing is due to an incorrect driver class name. 1. Correcting the JDBC Driver Class The error 
doesn't find class: com.starrocks.cj.jdbc.Driver
occurs because that class name does not exist. It looks like a mix-up with the MySQL driver class (
com.mysql.cj.jdbc.Driver
). If you are using the StarRocks JDBC Driver, the correct class name is: * Driver Class: 
com.starrocks.jdbc.Driver
* JDBC URI: 
jdbc:starrocks://<fe_host>:<query_port>/<database>
When creating a JDBC Catalog or JDBC Resource, ensure the 
driver_url
points to the correct JAR file and the 
driver_class
is exactly 
com.starrocks.jdbc.Driver
. 2. Recommended Method: Native StarRocks External Table For connecting two StarRocks clusters, using the StarRocks External Table (ENGINE=olap) is significantly more efficient than JDBC. It uses StarRocks' native RPC protocol (ports 9020/8060) instead of the MySQL/JDBC protocol, providing better performance and support for all StarRocks data types. Example Syntax:
Copy code
sql
CREATE EXTERNAL TABLE remote_table
(
    k1 DATE,
    k2 INT,
    v1 VARCHAR(2048)
)
ENGINE=olap
PROPERTIES
(
    "host" = "<remote_fe_host>",
    "port" = "9020",      -- Remote FE RPC port
    "user" = "root",
    "password" = "",
    "database" = "remote_db",
    "table" = "target_table"
);
3. Why the MySQL Driver might be "missing" tables If you use a JDBC Catalog with the MySQL driver to connect to StarRocks: * Table Types: The MySQL driver primarily looks for standard 
TABLE
types. If your remote StarRocks instance contains Materialized Views, the MySQL driver might not list them by default. * Schema Visibility: StarRocks' internal metadata often requires explicit database selection. If you are using a JDBC Catalog, ensure you use 
SHOW TABLES FROM <catalog_name>.<db_name>
. * Permissions: Ensure the user has 
SELECT
and 
USAGE
privileges on the remote database/catalog. Troubleshooting Checklist 1. JAVA_HOME: Ensure your BEs are using a JDK (not JRE). Check 
be/bin/start_be.sh
and ensure 
JAVA_HOME
is set correctly. 2. Driver Accessibility: The 
driver_url
you provide must be reachable by all FE and BE nodes. You can also place the JAR file manually in 
${STARROCKS_HOME}/lib/jdbc_drivers
on all nodes and use a 
file:///
URL. 3. Connectivity: Ensure the source cluster can reach the destination cluster's Query Port (9030) for JDBC, or RPC ports (9020/8060) for the native External Table. References • data_source/catalog/jdbc_catalog.md • data_source/catalog/default_catalog.md • sql-reference/sql-statements/table_bucket_part_index/SHOW_TABLES.md • data_source/feature-support-data-lake-analytics.md
If you are getting a "class not found" error for both 
com.starrocks.cj.jdbc.Driver
and 
com.starrocks.jdbc.Driver
, the problem is likely that the StarRocks cluster cannot physically find or load the JAR file you've specified, or there is a configuration issue with the Java environment. 1. Recommended Way: StarRocks-to-StarRocks For connecting two StarRocks clusters, do not use JDBC. It is significantly slower and less compatible than the native method. Instead, use a StarRocks External Table (
ENGINE=olap
). This uses StarRocks' native RPC protocol, supports all StarRocks data types, and is much faster. Example:
Copy code
sql
CREATE EXTERNAL TABLE remote_sr_table
(
    id INT,
    name VARCHAR(255)
)
ENGINE=olap
PROPERTIES
(
    "host" = "<remote_fe_host>",
    "port" = "9020",      -- Remote FE RPC port (default 9020)
    "user" = "root",
    "password" = "",
    "database" = "remote_db",
    "table" = "target_table"
);
────────── 2. Troubleshooting the JDBC Driver Class Error If you must use JDBC (e.g., for a JDBC Catalog), follow these steps to fix the "class not found" error: Check the Driver URL and Class Name * Class Name: The correct class for the native StarRocks Connector/J is 
com.starrocks.jdbc.Driver
. * Driver URL: Ensure you are providing a valid 
driver_url
in your 
CREATE EXTERNAL CATALOG
or 
RESOURCE
statement. The FE must be able to download this JAR. * Example: 
"driver_url" = "<https://repo1.maven.org/maven2/com/starrocks/starrocks-connector-j/1.1.1/starrocks-connector-j-1.1.1.jar>"
Verify BE Java Environment The JDBC scanner runs on the Backend (BE). A common cause for "class not found" is that the BE is using a JRE instead of a JDK. * Open 
be/bin/start_be.sh
on every BE node. * Ensure 
JAVA_HOME
points to a full JDK path. * Restart the BE after making changes. Check the Logs If it still fails, check the logs on the leader FE (
fe/log/fe.log
) and the BEs (
be/log/be.INFO
). Look for errors related to "Downloading driver" or "Failed to load class". ────────── 3. Why the MySQL Driver "misses" tables If you use the MySQL driver to connect to StarRocks, it might not show tables because: * Catalog Context: StarRocks has a three-tier structure (
catalog.database.table
). The MySQL driver often only sees 
default_catalog
. In your 
jdbc_uri
, try specifying the catalog: 
jdbc:mysql://<host>:<port>/default_catalog
. * Metadata Caching: If you just added tables, you may need to run 
REFRESH EXTERNAL TABLE <table_name>
or wait for the metadata cache to expire. * Table Types: Some drivers filter out non-base tables. Check if the "missing" tables are Materialized Views or Logic Views. Summary: Switch to 
ENGINE=olap
 for StarRocks-to-StarRocks connectivity. It bypasses JDBC drivers entirely and solves both the performance and "missing table" issues. References • data_source/catalog/jdbc_catalog.md • faq/Deploy_faq.md • data_source/External_table.md
Yes, 
ENGINE=olap
 (StarRocks External Table) or a StarRocks External Catalog is significantly better than JDBC for copying data between two StarRocks instances. JDBC is intended for general connectivity to third-party databases. For StarRocks-to-StarRocks operations, using the native protocol is faster, supports all StarRocks data types (like BITMAP, HLL, etc.), and is more reliable. Here are the three best ways to copy everything 1:1, depending on your scale: 1. The "Easiest" way: StarRocks External Catalog If you want to "see" every database and table from the remote cluster immediately without manual 
CREATE TABLE
statements, use a StarRocks Catalog. * Why it's better: You don't have to define the schema for every table. It automatically maps all remote tables. * Syntax:
Copy code
sql
CREATE EXTERNAL CATALOG remote_sr
PROPERTIES
(
    "type" = "starrocks",
    "starrocks.host" = "<remote_fe_host>",
    "starrocks.port" = "9020", -- FE RPC port
    "starrocks.user" = "root",
    "starrocks.password" = "password"
);
* Copying: You can then copy data simply: 
INSERT INTO local_db.table SELECT * FROM remote_sr.remote_db.table;
2. The "Native" way: StarRocks External Table (
ENGINE=olap
) If you only need to sync a few specific tables 1:1. * Why it's better: It uses the BE-to-BE native transmission protocol (Direct I/O), which is much faster than the MySQL/JDBC protocol. * Syntax: (As mentioned previously, ensure the schema matches exactly). 3. The "Heavyweight" way: Backup and Restore If you are copying terabytes of data or entire databases 1:1. * Process: 1. Create a Shared Repository (S3, HDFS, or MinIO) accessible by both clusters. 2. On Cluster A: 
BACKUP SNAPSHOT db_name.snapshot_name TO repository_name;
3. On Cluster B: 
RESTORE SNAPSHOT db_name.snapshot_name FROM repository_name;
* Why it's better: It copies the underlying data files directly. It is the most efficient method for a true 1:1 clone. 4. Automated Tool: StarRocks Cluster Sync StarRocks provides a specific tool called 
starrocks-cluster-sync
for migrating data between clusters. * It can automate the creation of schemas and the migration of data. * You can find it in the official documentation. Summary Recommendation * For SQL-based copying of multiple tables: Use the StarRocks External Catalog. It solves your "doesn't find class" error and the "missing tables" issue because it doesn't use the JDBC driver or the MySQL protocol for metadata. * For full cluster migration: Use Backup & Restore. References • administration/data_migration_tool.md • administration/management/Backup_and_restore.md • data_source/External_table.md • faq/operation_maintenance_faq.md References • data_source/catalog/unified_catalog.md • data_source/catalog/maxcompute_catalog.md • data_source/catalog/default_catalog.md • loading/InsertInto.md • data_source/External_table.md
Yes, it will work. StarRocks 3.5 and 4.1 are fully compatible for cross-cluster data access using native methods. In fact, using the native StarRocks External Catalog or 
ENGINE=olap
 is much safer for cross-version communication than JDBC, as they are designed to handle StarRocks-specific metadata and data types that a generic JDBC driver might misinterpret. Why this is the best path for 3.5 to 4.1: 1. Protocol Stability: The RPC protocol used by 
ENGINE=olap
(port 9020) is stable across these versions. StarRocks maintainers ensure backward compatibility for these native connections. 2. Feature Support: Version 4.1 introduced significant performance and architecture improvements (like enhanced tablet splitting). A StarRocks 4.1 cluster can query a 3.5 cluster as an external data source without any issues. 3. No Driver Headache: You won't have to deal with 
com.starrocks.jdbc.Driver
class-path errors or JAR versions, as the logic is built directly into the StarRocks binary. How to set it up (Recommended) Since you are copying "everything," the StarRocks External Catalog is your best choice. It will automatically map every database and table from your 3.5 instance into your 4.1 instance. Run this on your 4.1 cluster:
Copy code
sql
CREATE EXTERNAL CATALOG cluster_35
PROPERTIES
(
    "type" = "starrocks",
    "starrocks.host" = "<3.5_FE_IP>",
    "starrocks.port" = "9020", -- The RPC port of the 3.5 FE
    "starrocks.user" = "root",
    "starrocks.password" = "your_password"
);
Then copy data 11
Copy code
sql
-- View the tables from the 3.5 cluster
SHOW TABLES FROM cluster_35.remote_db;

-- Copy a table
INSERT INTO local_db.target_table 
SELECT * FROM cluster_35.remote_db.source_table;
Important Version Note If you are performing a one-time full migration (moving all data permanently from 3.5 to 4.1), you should use the StarRocks Migration Tool. It is a specialized binary designed to automate this exact process across versions. * StarRocks Migration Tool: Download and Docs * Back up and Restore: You can also take a snapshot on 3.5 and restore it on 4.1 using a shared S3/HDFS repository. Summary: Skip the JDBC driver. Use the StarRocks Catalog for interactive copying or the Migration Tool for a full 1:1 move. Both work perfectly between 3.5 and 4.1.
I apologize—you are correct. There is no native catalog type named 
"starrocks"
in the 
CREATE EXTERNAL CATALOG
command. I mixed this up with other systems. For connecting StarRocks 3.5 to StarRocks 4.1, you should use a JDBC Catalog with the standard MySQL driver. Since StarRocks is MySQL-compatible, the MySQL driver is the intended way to query one StarRocks cluster from another. 1. Correct Setup: JDBC Catalog using MySQL Driver This is the standard way to pull data (run 
SELECT
) from the 3.5 cluster into your 4.1 cluster. Using the MySQL driver avoids the classpath issues you had with the StarRocks-specific driver. Run this on your 4.1 cluster:
Copy code
sql
CREATE EXTERNAL CATALOG cluster_35
PROPERTIES
(
    "type" = "jdbc",
    "user" = "root",
    "password" = "your_password",
    -- Specify the catalog in the URI to avoid "missing tables"
    "jdbc_uri" = "jdbc:mysql://<3.5_FE_IP>:9030/default_catalog",
    "driver_url" = "<https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.28/mysql-connector-java-8.0.28.jar>",
    "driver_class" = "com.mysql.cj.jdbc.Driver"
);
* Why use 
default_catalog
in the URI? StarRocks uses a 3-tier structure (
catalog.db.table
). Some MySQL drivers only see the "default" level. Explicitly adding 
/default_catalog
to the URI often solves the "missing tables" problem. * Version Compatibility: This method works perfectly between 3.5 and 4.1. ────────── 2. Alternative: StarRocks External Table (
ENGINE=olap
) If you want to push data from 3.5 to 4.1 (i.e., you are logged into the 3.5 cluster and want to send data to the new 4.1 cluster), you use the native 
ENGINE=olap
. Note: You cannot run 
SELECT
on these tables. They are "write-only" endpoints used for 
INSERT INTO
. On the 3.5 Cluster (Source):
Copy code
sql
CREATE EXTERNAL TABLE push_to_41
(
    id INT,
    name VARCHAR(255)
)
ENGINE=olap
PROPERTIES
(
    "host" = "<4.1_FE_IP>",
    "port" = "9020",      -- FE RPC port
    "user" = "root",
    "password" = "your_password",
    "database" = "target_db",
    "table" = "target_table"
);

-- Copy data
INSERT INTO push_to_41 SELECT * FROM local_table;
────────── 3. For a Full 1:1 Migration If you are moving everything and want it to be identical, the StarRocks Migration Tool is the official recommendation. It automates the schema and data copying and is designed to handle version jumps like 3.5 to 4.1. * Download: StarRocks Migration Tool Summary * To Pull (SELECT): Use JDBC Catalog with 
com.mysql.cj.jdbc.Driver
. * To Push (INSERT): Use 
ENGINE=olap
 External Tables. * For Full Migration: Use the Migration Tool. References • data_source/catalog/jdbc_catalog.md • data_source/External_table.md • administration/data_migration_tool.md References • data_source/catalog/jdbc_catalog.md • data_source/External_table.md
4 Views
Open in Slack
PreviousNext
Powered by