http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> StarRocks cluster 3.5.13 CREATE STO...
# questions-and-troubleshooting
p
@Rocky StarRocks cluster 3.5.13 CREATE STORAGE VOLUME gcs_new TYPE = GS LOCATIONS = ("gs://my-gcs-bucket/starrocks") PROPERTIES ( "gcp.gcs.use_compute_engine_service_account" = "true" ); SET gcs_new AS DEFAULT STORAGE VOLUME; why new create table still use built_in_storage volume create table ops.test_new_buckets3 (id int) ; show create table ops.test_new_buckets3; CREATE TABLE 
test_new_buckets3
( 
id
int(11) NULL COMMENT "" ) ENGINE=OLAP DUPLICATE KEY(
id
) COMMENT "OLAP" DISTRIBUTED BY RANDOM PROPERTIES ( "bucket_size" = "4294967296", "compression" = "LZ4", "datacache.enable" = "true", "enable_async_write_back" = "false", "replication_num" = "1", "storage_volume" = "builtin_storage_volume" );
r
The behavior you are seeing is due to the inheritance hierarchy of storage volumes in StarRocks. When you create a table, it inherits its 
storage_volume
property in this order: 1. Table level (if specified in 
PROPERTIES
) 2. Database level (if specified during 
CREATE DATABASE
or 
ALTER DATABASE
) 3. Cluster/Catalog default Why it happened If your database 
ops
was created before you set 
gcs_new
as the default storage volume, the database likely inherited the previous default (
builtin_storage_volume
). Even if you change the cluster-wide default later, existing databases retain the storage volume they were assigned at creation time. Because your new table 
ops.test_new_buckets3
belongs to the 
ops
database, it inherits the database's storage volume. How to fix it You need to update the storage volume property for the database 
ops
so that all future tables created within it use your new GCS volume. 1. Update the database's storage volume:
Copy code
sql
    ALTER DATABASE ops SET ("storage_volume" = "gcs_new");
2. Verify the change: You can check if the database property has been updated:
Copy code
sql
    SHOW CREATE DATABASE ops;
3. Test with a new table: Now, create a new table and verify its storage volume:
Copy code
sql
    CREATE TABLE ops.test_inheritance (id int) DISTRIBUTED BY RANDOM;
    SHOW CREATE TABLE ops.test_inheritance;
Important Notes * Existing Tables: Altering the database's storage volume only affects new tables. Existing tables (like your 
test_new_buckets3
) will continue to use the volume they were created with. If you need an existing table to use the new volume, you must recreate the table (e.g., 
INSERT INTO ... SELECT
). * Built-in Storage Volume: The 
builtin_storage_volume
is the default internal volume created by StarRocks (usually pointing to the path defined in your 
fe.conf
during initial setup). * Privileges: Ensure you have the 
ALTER
privilege on the database and the 
USAGE
privilege on the 
gcs_new
storage volume. Reference: ALTER DATABASE | Storage Volumes References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • sql-reference/sql-statements/cluster-management/storage_volume/ALTER_STORAGE_VOLUME.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md • sql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md • faq/Others.md
Yes, StarRocks provides several metrics that can be used to monitor failures in statistics collection and internal queries. Depending on your StarRocks version and what specifically you want to monitor, you can use the following metrics in Grafana: 1. Internal Query Failure Metric Statistics collection in StarRocks is executed as internal SQL queries. Failures for these internal operations (including background 
ANALYZE
tasks) are tracked by: * Metric Name: 
starrocks_fe_query_internal_err
* Description: Tracks the number of internal queries that resulted in an error. * PromQL for Alerting:
Copy code
promql
    # Alert if there are more than 5 internal query failures in a 5-minute window
    increase(starrocks_fe_query_internal_err[5m]) > 5
Note: This metric is available in newer versions (v3.1.15+, v3.2.11+, v3.3.3+). 2. General Statistics Collection Metrics StarRocks FEs export specific counters for the statistics (CBO) module. These are usually visible at your FE's 
/metrics
endpoint: * 
starrocks_fe_statistic_collect_total
: The total number of statistics collection tasks attempted. * 
starrocks_fe_statistic_collect_failed_total
: The total number of failed statistics collection tasks. * PromQL for Failure Rate:
Copy code
promql
    # Percentage of failed stats tasks over the last hour
    rate(starrocks_fe_statistic_collect_failed_total[1h]) / rate(starrocks_fe_statistic_collect_total[1h]) * 100
3. Monitoring via SQL (Alternative) If you want to create an alert based on the status of specific analysis tasks, you can query the internal metadata tables directly. While not a Grafana metric by default, you can use a Prometheus SQL exporter to turn these into metrics:
Copy code
sql
-- Check for failed analyze jobs in the last 24 hours
SELECT * FROM _statistics_.analyze_status 
WHERE state = 'FAILED' 
  AND end_time > NOW() - INTERVAL 1 DAY;
4. Log-based Monitoring For detailed troubleshooting of statistics failures, StarRocks provides a specific log file: * Log Path: 
fe/log/fe.internal.log
* Module: Look for entries tagged with 
internal.statistic
. This log is dedicated to recording internal SQL executions and the statistics collection process. Summary Checklist for Grafana To set up a robust alert for statistics: 1. Alert on 
starrocks_fe_query_internal_err
 for immediate notification of system query failures. 2. Monitor 
starrocks_fe_statistic_collect_failed_total
 to track the health of the Cost-Based Optimizer's background tasks. 3. Check 
fe.audit.log
 for any 
ANALYZE
statements that have a 
State=ERR
to see the exact SQL and specific error message. References • administration/management/monitoring/Monitor_and_Alert.md • administration/management/monitoring/alert.md • administration/management/monitoring/metrics.md • administration/management/FE_configuration.md • faq/Others.md References • administration/management/monitoring/metrics.md • administration/management/monitoring/Monitor_and_Alert.md • administration/management/monitoring/metrics-materialized_view.md • administration/management/BE_configuration.md
p
@Rocky which starrocks version support change storage volume of database ALTER DATABASE ops SET ("storage_volume" = "gcs_new");
r
The 
ALTER DATABASE <db_name> SET ("storage_volume" = "<volume_name>");
statement is supported starting from StarRocks version 3.1. Key Version Details: * Version 3.1.0: Introduced the Storage Volume feature for shared-data (compute-storage separated) clusters. This included the ability to create, alter, and set default storage volumes, as well as assigning them at the database level during creation or via 
ALTER DATABASE
. * Inheritance Logic: In version 3.1 and later, StarRocks follows a strict inheritance hierarchy: 
Table
> 
Database
> 
Default Catalog
. Changing the volume at the database level ensures all newly created tables in that database inherit the new volume. Usage Notes for v3.1+: * Shared-Data Mode only: This syntax and the use of storage volumes are specific to StarRocks clusters running in shared-data mode (where data is stored in object storage or HDFS). * Existing Tables: As you noted in your previous interaction, 
ALTER DATABASE
only changes the default for future tables. It does not migrate the data of existing tables to the new storage volume. * Privileges: To run this command, you must have the 
ALTER
privilege on the database and the 
USAGE
privilege on the specific storage volume. References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md • deployment/shared_data/feature-support-shared-data.md • sql-reference/sql-statements/cluster-management/storage_volume/SET_DEFAULT_STORAGE_VOLUME.md References • deployment/shared_data/feature-support-shared-data.md • release_notes/release-2.5.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md • release_notes/release-3.4.md • knowledge/usage/storage/hdfs_storage_volume_extended.md • data_source/feature-support-data-lake-analytics.md
p
@Rocky ALTER DATABASE ops SET (storage_volume = "gcs_new"); (pymysql.err.ProgrammingError) (1064, "Getting syntax error at line 1, column 23. Detail message: No viable statement for input 'ALTER DATABASE ops SET ('.") [SQL: ALTER DATABASE ops SET (storage_volume = "gcs_new")] (Background on this error at: sqlalche.me/e/14/f405)
r
The syntax error you encountered is because StarRocks requires the property name 
storage_volume
to be enclosed in double quotes within the 
SET
clause. The Correct Syntax To fix the error, update your command to include quotes for both the property key and the volume name:
Copy code
sql
ALTER DATABASE ops SET ("storage_volume" = "gcs_new");
Why the error occurred * Property Keys as Strings: In StarRocks, when using the 
SET ("key" = "value")
syntax for database or table properties, the key (in this case 
storage_volume
) must be a quoted string. * Parser Strictness: The error message 
No viable statement for input 'ALTER DATABASE ops SET ('
indicates the parser was expecting a quoted string after the opening parenthesis, but it encountered the unquoted identifier 
storage_volume
instead. Troubleshooting Checklist If you still see errors after adding quotes, please verify the following: 1. Privileges: Ensure your user has the 
ALTER
privilege on the database 
ops
and the 
USAGE
privilege on the storage volume 
gcs_new
. 2. Volume Existence: Run 
SHOW STORAGE VOLUMES;
to confirm that 
gcs_new
exists and is 
ENABLED
. 3. Shared-Data Mode: Ensure your cluster is running in shared-data (compute-storage separated) mode, as Storage Volumes are not used in shared-nothing (local storage) mode. References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md • sql-reference/sql-statements/account-management/GRANT.md References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • sql-reference/sql-statements/cluster-management/storage_volume/ALTER_STORAGE_VOLUME.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md • sql-reference/sql-statements/account-management/GRANT.md
p
@Rocky ALTER DATABASE ops SET ("storage_volume" = "gcs_new"); (pymysql.err.ProgrammingError) (1064, "Getting syntax error at line 1, column 23. Detail message: No viable statement for input 'ALTER DATABASE ops SET ('.") [SQL: ALTER DATABASE ops SET ("storage_volume" = "gcs_new")] (Background on this error at: https://sqlalche.me/e/14/f405)
r
The syntax error you are receiving (
No viable statement for input 'ALTER DATABASE ops SET ('
) indicates that your StarRocks cluster's parser does not recognize the property-setting syntax for 
ALTER DATABASE
. This most commonly happens for one of the following two reasons: 1. You are on a version older than 3.1 The ability to set a 
storage_volume
for a database was introduced in StarRocks version 3.1.0. * If you are running StarRocks 2.x or 3.0, the parser only knows how to handle 
SET DATA QUOTA
or 
SET REPLICA QUOTA
. It does not expect an opening parenthesis 
(
after the 
SET
keyword, which is why it fails at that exact character. * Action: Run 
SELECT VERSION();
to verify your version. If it is lower than 3.1, you cannot use storage volumes. 2. The Cluster is in Shared-Nothing Mode Storage Volumes are a feature specifically for Shared-Data (Compute-Storage Separated) clusters. * If your StarRocks cluster is a standard Shared-Nothing cluster (where data is stored on local disks of BEs), the storage volume functionality is not applicable. * While the parser in newer versions usually recognizes the keywords regardless of mode, some earlier builds of 3.1/3.2 might have stricter parsing if the cloud-native features aren't enabled. Possible Workaround/Alternative Syntaxes If you are certain you are on v3.1+ and the error persists, try these variations to see if your specific sub-version has a slightly different parser rule: Option A: Without Quotes on the property key (rare, but sometimes required in specific client environments):
Copy code
sql
ALTER DATABASE ops SET (storage_volume = "gcs_new");
Option B: Using single quotes (Standard SQL style):
Copy code
sql
ALTER DATABASE ops SET ('storage_volume' = 'gcs_new');
Option C: Using PROPERTIES keyword (Used in some older internal versions):
Copy code
sql
ALTER DATABASE ops SET PROPERTIES ("storage_volume" = "gcs_new");
Recommendation If none of the above work, it is highly likely that your version of StarRocks does not yet support this command. To achieve a similar result on versions that don't support 
ALTER DATABASE ... SET STORAGE_VOLUME
, you would need to: 1. Set the volume at table creation: Explicitly define the 
storage_volume
property in each 
CREATE TABLE
statement. 2. Set a Default Volume: Use 
SET <volume_name> AS DEFAULT STORAGE VOLUME;
to change the default for the entire cluster (this will affect all new databases and tables). References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md • sql-reference/sql-statements/cluster-management/storage_volume/SET_DEFAULT_STORAGE_VOLUME.md • deployment/shared_data/feature-support-shared-data.md References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • sql-reference/sql-statements/cluster-management/storage_volume/ALTER_STORAGE_VOLUME.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md
k
@Rocky update your knowledge, the update storage volume property for database is implemented in PR: https://github.com/StarRocks/starrocks/pull/67699, supported version 4.0.5+ and 4.1+
r
Thank you for the correction. You are absolutely right. While earlier documentation mentioned the concept of storage volumes, the specific ability to update the 
storage_volume
property of an existing database using the 
ALTER DATABASE
statement was not implemented until Pull Request #67699. As you noted, this feature is available in: * StarRocks 4.0.5 and later * StarRocks 4.1 and later Summary of the implementation: In versions prior to these (including all 3.x releases), the 
ALTER DATABASE
command only supports setting data and replica quotas. Attempting to use the 
SET ("storage_volume" = ...)
syntax results in the 
1064
syntax error you encountered because the parser does not yet recognize that property for databases. For users on older versions: If you are on a version earlier than 4.0.5/4.1, your options are: 1. Set at Creation: Specify the 
storage_volume
only during the 
CREATE DATABASE
phase. 2. Default Volume: Use 
SET <volume_name> AS DEFAULT STORAGE VOLUME;
to change the cluster-wide default for all new objects. 3. Table Level: Manually specify the 
storage_volume
property for each individual table during its creation. References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • release_notes/release-4.0.md • sql-reference/sql-statements/Database/CREATE_DATABASE.md References • sql-reference/sql-statements/Database/ALTER_DATABASE.md • sql-reference/sql-statements/Database/CREATE_DATABASE.md • release_notes/release-3.1.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md • sql-reference/sql-statements/cluster-management/storage_volume/SET_DEFAULT_STORAGE_VOLUME.md
3 Views
Open in Slack
PreviousNext
Powered by