http://starrocks.io logo
Join Slack
Powered by
This message was deleted.
# questions-and-troubleshooting
s
This message was deleted.
r
In StarRocks, compute isolation is achieved differently depending on your deployment architecture. If you are looking for Snowflake-style isolated warehouses, you should look into the Shared-data (Compute-Storage Separation) architecture introduced in version 3.0, specifically the Multi-Warehouse (Compute Groups) feature added in version 3.3. 1. Multi-Warehouse (Shared-data Architecture) In a shared-data deployment, you can create multiple isolated warehouses (compute groups) that share the same underlying storage (e.g., S3, GCS) but use independent sets of Compute Nodes (CN). * Create a Warehouse:
Copy code
sql
    CREATE WAREHOUSE <warehouse_name>;
* Switch Warehouse: You can specify which warehouse to use for your current session:
Copy code
sql
    SET WAREHOUSE = '<warehouse_name>';
* Manage/View Warehouses:
Copy code
sql
    SHOW WAREHOUSES;
    SELECT current_warehouse();
* Benefits: This provides full physical isolation. Queries in one warehouse do not compete for CPU or Memory with queries in another. 2. Resource Groups (Internal Isolation) If you are using a Shared-nothing architecture (standard BE nodes) or want more granular isolation within a single warehouse, you use Resource Groups. This feature allows you to isolate workloads by limiting CPU and memory at the software level. * Create a Resource Group:
Copy code
sql
    CREATE RESOURCE GROUP <group_name>
    TO (user='sales_user', role='analyst') -- Classifiers
    WITH (
        "exclusive_cpu_cores" = "10", -- Hard CPU isolation (v3.3.5+)
        "mem_limit" = "20%",           -- Memory limit
        "concurrency_limit" = "10"    -- Max concurrent queries
    );
* Documentation Reference: You can find more details in the official docs under Administration Resource Management > Resource Group>. Key Differences Feature | Isolation Level | Deployment Mode | Version Warehouse | Physical (Node-level) | Shared-data | 3.3+ Resource Group | Logical (CPU/Mem limits) | Both | 2.2+ Note: The 
CREATE WAREHOUSE
syntax and management are primarily available for shared-data clusters. If you are running a standard shared-nothing cluster, Resource Groups are the primary way to achieve isolation. References • faq/shared_data_faq.md • quick_start/shared-data.md • deployment/shared_data/gcs.md • administration/management/BE_configuration.md • deployment/shared_data/azure.md
z
@Rocky yes I use shared arch and give me warehouse creation and manage doc
r
In StarRocks, the Multi-Warehouse (Compute Groups) feature for shared-data architecture was introduced in version 3.3. This allows you to create multiple isolated compute clusters (warehouses) that share the same underlying storage. 1. Creating and Managing Warehouses You can manage warehouses directly via SQL commands. * Create a Warehouse:
Copy code
sql
    CREATE WAREHOUSE <warehouse_name>;
* Show all Warehouses:
Copy code
sql
    SHOW WAREHOUSES;
* Drop a Warehouse:
Copy code
sql
    DROP WAREHOUSE <warehouse_name>;
2. Assigning Compute Nodes (CN) to a Warehouse A warehouse is essentially a group of Compute Nodes (CN). When you add a new CN to the cluster, you can specify which warehouse it belongs to. * Add a node to a specific warehouse:
Copy code
sql
    ALTER SYSTEM ADD COMPUTE NODE "host:port" WAREHOUSE "warehouse_name";
* View nodes and their warehouses:
Copy code
sql
    SHOW COMPUTE NODES;
This will show a 
Warehouse
column indicating the assignment. 3. Using a Warehouse Workload isolation is achieved by switching the session context or routing queries. * Switch current session to a warehouse:
Copy code
sql
    SET WAREHOUSE = '<warehouse_name>';
* Verify current warehouse:
Copy code
sql
    SELECT current_warehouse();
4. Advanced Management * Workload Isolation: Since each warehouse uses its own physical CN nodes, a heavy query in 
warehouse_a
will not impact the CPU/Memory performance of 
warehouse_b
. * Compaction Isolation: You can designate a specific warehouse for background tasks like compaction to ensure they don't interfere with user queries.
Copy code
sql
    -- Example: Setting a warehouse for specific tasks (v3.3+)
    SET GLOBAL compaction_warehouse = 'compaction_wh';
Documentation References Detailed documentation for these commands is typically found in the SQL Reference or Administration sections for version 3.3 and later. • StarRocks v3.3 Release Notes • Compute-Storage Separation Overview Note: If you are using a managed service (like StarRocks Cloud or CelerData), these warehouses are often managed via a UI, but the SQL commands above remain functional for session management. References • introduction/Architecture.md • quick_start/shared-data.md • deployment/shared_data/s3.md • deployment/shared_data/gcs.md • faq/shared_data_faq.md References • quick_start/shared-data.md • deployment/shared_data/gcs.md
z
@Rocky just link the doc where there is explained multi-warehouse compute groups
r
For StarRocks shared-data architecture, the feature you are looking for is called Multi-Warehouse (Compute Groups), introduced in version 3.3. This allows you to create multiple isolated compute clusters that share the same underlying storage (S3, GCS, etc.) for physical workload isolation. Documentation Link The primary documentation for managing these warehouses can be found here: * *Compute Group (Multi-Warehouse)* Key Commands for Isolated Warehouses Once you have a shared-data cluster (3.3+), you can use the following SQL commands to manage your warehouses: * Create a new isolated warehouse:
Copy code
sql
    CREATE WAREHOUSE <warehouse_name>;
* Assign Compute Nodes (CN) to a warehouse: When adding nodes, you can bind them to a specific warehouse to ensure physical isolation:
Copy code
sql
    ALTER SYSTEM ADD COMPUTE NODE "host:port" WAREHOUSE "warehouse_name";
* Switch to a specific warehouse: Queries in your current session will only run on the specified warehouse:
Copy code
sql
    SET WAREHOUSE = '<warehouse_name>';
* View status and metrics:
Copy code
sql
    SHOW WAREHOUSES; -- List all warehouses
    SELECT * FROM information_schema.warehouse_metrics; -- Performance metrics per warehouse
    SELECT * FROM information_schema.warehouse_queries; -- Active queries per warehouse
Why use this over Resource Groups? * Warehouse: Provides physical isolation. Different warehouses use different sets of Compute Nodes (CNs). A heavy query in one warehouse cannot steal CPU or Memory from another. * Resource Group: Provides logical isolation within a single warehouse/set of nodes using soft/hard limits on CPU and Memory. If you are looking for a Snowflake-like experience where you have "Small", "Medium", or "Large" clusters for different departments, Multi-Warehouse is the correct feature. References • administration/management/resource_management/resource_group.md
z
@Rocky this link is not availablke docs.starrocks.io/docs/…/warehouse did you just drop the warehouse concept?
r
The warehouse concept has not been dropped. In StarRocks v3.3 and later (Shared-data architecture), it is the primary way to achieve physical compute isolation. The link you provided might be broken due to a recent documentation reorganization where "Warehouses" are often referred to as Compute Groups in the management section. Correct Documentation Links Depending on your version, you can find the management details here: * *Compute Group (Multi-warehouse) Management* * *Warehouse Information Schema* (to monitor active queries) Summary of Warehouse Management If you are on a shared-data cluster (Compute-Storage Separation), use these commands to manage isolated warehouses: 1. Create an isolated warehouse:
Copy code
sql
    CREATE WAREHOUSE <warehouse_name>;
2. Assign nodes to the warehouse: When adding a Compute Node (CN), bind it to the warehouse to ensure physical isolation.
Copy code
sql
    ALTER SYSTEM ADD COMPUTE NODE "host:port" WAREHOUSE "warehouse_name";
3. Switch to a specific warehouse in your session: All subsequent queries in the session will run only on the nodes assigned to this warehouse.
Copy code
sql
    SET WAREHOUSE = '<warehouse_name>';
4. Verify current warehouse usage:
Copy code
sql
    SELECT current_warehouse();
    SHOW WAREHOUSES;
Why you might see a 404 StarRocks recently updated its documentation structure for the 3.x and 4.0 release cycles. If 
.../compute_node/warehouse/
is failing, it is likely because the page was renamed to Compute Group (
.../compute_node/compute_group/
) to better reflect the underlying architecture of grouping Compute Nodes. References • sql-reference/information_schema/warehouse_queries.md • sql-reference/sql-functions/utility-functions/current_warehouse.md • sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_COMPUTE_NODES.md • faq/shared_data_faq.md References • administration/management/resource_management/resource_group.md
2 Views
Open in Slack
PreviousNext
Powered by