< Rocky> how to separate Cn nodes for ingestion vs CN nodes StarRocks #questions-and-troubleshooting

Join Slack

<@U0A71G31CDV> how to separate Cn nodes for ingest...

# questions-and-troubleshooting

Andrei Harbunou

06/08/2026, 9:13 PM

@Rocky how to separate Cn nodes for ingestion vs CN nodes that servers query using warehouse?

Rocky

06/08/2026, 9:14 PM

In StarRocks (starting from version 3.3+), you can separate ingestion and query workloads using the Warehouse (or Computing Group) feature. This capability is specifically available in shared-data (storage-compute separation) deployments. By creating multiple warehouses, you can dedicate specific Compute Nodes (CNs) to ingestion and others to serving queries, ensuring that heavy data loading does not impact query latency. 1. Create Dedicated Warehouses First, you define the warehouses in your cluster. For example, you can create one for ingestion and one for ad-hoc queries:

Copy code

sql
-- Create a warehouse for ingestion
CREATE WAREHOUSE ingestion_wh;

-- Create a warehouse for queries
CREATE WAREHOUSE query_wh;

2. Assign CN Nodes to Warehouses The method of assigning nodes depends on your deployment environment: * Kubernetes (Operator): You define separate

StarRocksWarehouse

custom resources in your YAML. Each warehouse will have its own set of replicas (CN pods). * Manual/Bare Metal: When adding a CN node to the cluster, you can specify its warehouse:

Copy code

sql
    ALTER SYSTEM ADD COMPUTE NODE "host:port" WAREHOUSE ingestion_wh;

_Note: If you have already added nodes, they reside in the

default_warehouse

by default. You can view assignments using

SHOW COMPUTE NODES

._ 3. Route Workloads to Specific Warehouses For Ingestion (Stream Load) To ensure a load job runs on the ingestion-dedicated nodes, specify the warehouse in the HTTP header:

Copy code

bash
curl --location-trusted -u user:pass \
    -H "warehouse: ingestion_wh" \
    -T data.csv \
    <http://fe_host:http_port/api/db/table/_stream_load>

For Routine Load When creating a Routine Load job (e.g., from Kafka), specify the warehouse in the `PROPERTIES`:

Copy code

sql
CREATE ROUTINE LOAD my_job ON my_table
PROPERTIES (
    "warehouse" = "ingestion_wh"
)
FROM KAFKA (...);

For Queries In your SQL session or application connection, set the active warehouse to route queries to the query-dedicated nodes:

Copy code

sql
-- Set the warehouse for the current session
SET warehouse query_wh;

-- Check which warehouse is currently active
SELECT CURRENT_WAREHOUSE();

Key Considerations * Version Requirement: Ensure you are on StarRocks v3.3 or later. * Shared Data Mode: This feature requires the

run_mode = shared_data

configuration. * Isolation: Since data is stored centrally in object storage (S3/GCS/MinIO), all warehouses can access the same data simultaneously without data movement, but their CPU and memory resources remain physically isolated. References • sql-reference/sql-statements/cluster-management/resource_group/CREATE_RESOURCE_GROUP.md • sql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.md • sql-reference/sql-statements/Database/CREATE_DATABASE.md • sql-reference/sql-statements/Resource/CREATE_RESOURCE.md

Andrei Harbunou

06/08/2026, 9:17 PM

@Rocky I have only MVs, is it possible to separate nodes for MV ingestion?

Rocky

06/08/2026, 9:17 PM

Yes, you can separate CN nodes for Materialized View (MV) ingestion (refresh tasks) from the nodes serving queries by using the Warehouse (also known as Computing Group) feature. This capability is available in Shared-data mode (storage-compute separation) starting from StarRocks v3.3. 1. Create Dedicated Warehouses First, define a warehouse for ingestion and another for queries. This physically separates the Compute Nodes (CNs).

Copy code

sql
-- Create a warehouse dedicated to background MV refreshes
CREATE WAREHOUSE ingestion_wh;

-- Create a warehouse dedicated to user queries
CREATE WAREHOUSE query_wh;

2. Assign the MV Refresh Task to the Ingestion Warehouse When creating an asynchronous MV, you can specify which warehouse should handle the background refresh tasks using the

warehouse

property.

Copy code

sql
CREATE MATERIALIZED VIEW my_mv
REFRESH ASYNC EVERY(INTERVAL 1 HOUR)
PROPERTIES (
    "warehouse" = "ingestion_wh"
)
AS 
SELECT ... ;

3. Update Existing MVs If you already have MVs and want to move their refresh workloads to a specific warehouse, use the

ALTER

statement:

Copy code

sql
ALTER MATERIALIZED VIEW my_mv SET ("warehouse" = "ingestion_wh");

How the Separation Works * Ingestion (Refresh): The StarRocks FE will schedule the MV refresh job on the CN nodes belonging to

ingestion_wh

. This ensures that heavy calculations during the refresh do not steal CPU or memory from your query nodes. * Queries: When a user or application runs a query, the session warehouse (typically

query_wh

) is used. Even if the query is transparently rewritten to use the MV, the query execution itself happens on the

query_wh

nodes. Alternative: Resource Groups If you are not using shared-data mode (or are on an older version like v3.1+), you can use Resource Groups for "soft" isolation. You can assign an MV to a resource group to limit its CPU and memory usage:

Copy code

sql
CREATE MATERIALIZED VIEW my_mv
PROPERTIES (
    "resource_group" = "low_priority_ingestion_group"
)
AS SELECT ...;

However, for physical node separation, the Warehouse feature in v3.3+ (Shared-data) is the recommended approach. References • sql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md • using_starrocks/async_mv/Materialized_view.md • using_starrocks/async_mv/use_cases/data_modeling_with_materialized_views.md • using_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.md • sql-reference/sql-statements/Resource/CREATE_RESOURCE.md • sql-reference/sql-statements/Database/CREATE_DATABASE.md • sql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.md • using_starrocks/async_mv/use_cases/create_partitioned_materialized_view.md References • sql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md

Andrei Harbunou

06/08/2026, 9:21 PM

@Rocky what I need for this?

Copy code

Queries: When a user or application runs a query, the session warehouse (typically query_wh) is used. Even if the query is transparently rewritten to use the MV, the query execution itself happens on the query_wh nodes.

how to set query_wh as default for queries?

Rocky

06/08/2026, 9:21 PM

To set a specific warehouse (like

query_wh

) as the default for queries, you have three primary methods depending on whether you want to set it for a single session, a specific user, or globally. 1. Per-User Default (Recommended) Starting from StarRocks v3.3.3, you can bind a session variable to a user property. This ensures that every time the user logs in, their session is automatically set to use the specified warehouse.

Copy code

sql
-- Replace 'jack' with your actual username
ALTER USER 'jack' SET PROPERTIES ('session.warehouse' = 'query_wh');

2. Session-Level (Manual) Any user can switch their warehouse at the start of a session. This is common for ad-hoc analysis or when using a BI tool where you can provide an "initialization SQL".

Copy code

sql
-- Set the warehouse for the current connection
SET WAREHOUSE query_wh;

-- Verify the setting
SELECT current_warehouse();

3. Global Default If you want to change the default warehouse for all users in the cluster who haven't specified one:

Copy code

sql
SET GLOBAL warehouse = 'query_wh';

Note: This only applies to new connections created after this command is executed. Existing connections will maintain their previous warehouse setting. 4. Application/JDBC Level If you are connecting via an application, you can often execute a

SET

command immediately after the connection is established. Some JDBC drivers also allow executing a "session init" script. Important: Permissions Before a user can use

query_wh

, they must be granted the

USAGE

privilege on that warehouse by an administrator:

Copy code

sql
GRANT USAGE ON WAREHOUSE query_wh TO USER 'jack';

Summary of Workflow 1. Create the warehouses (

ingestion_wh

and

query_wh

). 2. Assign CN nodes to them (via SQL or Kubernetes Operator). 3. Grant usage permissions to the relevant users/roles. 4. Configure the default for users via

ALTER USER ... SET PROPERTIES

. References • sql-reference/sql-functions/utility-functions/current_warehouse.md • sql-reference/System_variable.md • sql-reference/sql-statements/account-management/ALTER_USER.md • sql-reference/sql-statements/account-management/GRANT.md References • sql-reference/sql-functions/utility-functions/current_warehouse.md • sql-reference/System_variable.md • sql-reference/sql-statements/Catalog/SET_CATALOG.md

Andrei Harbunou

06/15/2026, 9:08 PM

@Rocky I'm on 4.0.11 and I see error

Multi-Warehouse is not implemented

when try to create new warehouse

Rocky

06/15/2026, 9:08 PM

The error

Multi-Warehouse is not implemented

typically occurs because the Multi-Warehouse (Computing Group) feature is exclusively supported in shared-data mode (storage-compute separation). If you are running StarRocks in the default shared-nothing mode (where data is stored on local BE disks), the

CREATE WAREHOUSE

statement is not available, even in version 4.0.11. How to check your mode Run the following command to verify your cluster's run mode:

Copy code

sql
ADMIN SHOW FRONTEND CONFIG LIKE 'run_mode';

* If it returns

shared_nothing

, you cannot use warehouses. * If it returns

shared_data

, ensure your CN nodes are properly initialized as compute nodes. ────────── How to achieve isolation in Shared-Nothing (v4.0.11) Since you cannot use warehouses in shared-nothing mode, you should use Resource Groups and BE Tags to separate your ingestion and query nodes. 1. Tag your BE nodes Assign a specific tag to the nodes you want to use for ingestion and a different tag for query nodes.

Copy code

sql
-- On nodes intended for ingestion
ALTER SYSTEM MODIFY BACKEND "host:port" SET ("tag.location" = "ingestion_node");

-- On nodes intended for queries
ALTER SYSTEM MODIFY BACKEND "host:port" SET ("tag.location" = "query_node");

2. Create Resource Groups Create resource groups that map to these specific tags.

Copy code

sql
CREATE RESOURCE GROUP ingestion_group
PROPERTIES (
    "cpu_core_limit" = "10",
    "mem_limit" = "0.5",
    "concurrency_limit" = "20"
);

3. Bind MVs to specific nodes In shared-nothing mode, the data itself is partitioned across BEs. However, you can control where the query execution or MV refresh happens using session variables or resource group classifiers. For MV refreshes, you can limit the resource consumption using:

Copy code

sql
ALTER MATERIALIZED VIEW my_mv SET ("resource_group" = "ingestion_group");

Summary * Warehouses are for Shared-Data mode (Cloud Native). * Resource Groups + Tags are for Shared-Nothing mode. If you specifically need the Warehouse feature, you would need to redeploy the cluster in

shared_data

mode using object storage (S3, MinIO, etc.) as the primary storage. References • data_source/catalog/unified_catalog.md • integrations/dbt.md • faq/Deploy_faq.md • faq/shared_data_faq.md • using_starrocks/async_mv/use_cases/query_rewrite_with_materialized_views.md • release_notes/release-3.1.md • sql-reference/sql-functions/utility-functions/current_warehouse.md

Andrei Harbunou

06/15/2026, 9:09 PM

@Rocky it says

shared_data

when I run

ADMIN SHOW FRONTEND CONFIG *LIKE* 'run_mode';

Rocky

06/15/2026, 9:10 PM

The error Multi-Warehouse is not implemented
confirms that you are using the Open Source (Community) Edition of StarRocks. While the "shared-data" mode (storage-compute separation) is available in the open-source version, the Multi-Warehouse (Computing Group) feature—which allows you to create multiple named compute clusters (

CREATE WAREHOUSE

) and switch between them—is an Enterprise Edition feature. In the Open Source version 4.0.11, you have a single pool of compute nodes (CNs) in your

shared_data

cluster. However, you can still achieve your goal of separating/isolating MV ingestion from user queries using Resource Groups. Solution for Open Source: Resource Group Isolation Since you cannot physically split nodes into different warehouses, you use "Soft Isolation" to ensure MV refreshes don't starve your queries. 1. Create a Resource Group for MV Refreshes Define a group that limits how much CPU and memory the MV background tasks can consume.

Copy code

sql
CREATE RESOURCE GROUP mv_refresh_group
PROPERTIES (
    "cpu_core_limit" = "10",      -- Max CPU cores this group can use
    "mem_limit" = "0.3",          -- Max 30% of node memory
    "concurrency_limit" = "5"     -- Max 5 concurrent refresh tasks
);

2. Bind your Materialized Views to this Group When creating or altering your MV, assign it to the resource group.

Copy code

sql
-- For a new MV
CREATE MATERIALIZED VIEW my_mv
REFRESH ASYNC
PROPERTIES (
    "resource_group" = "mv_refresh_group"
)
AS SELECT ...;

-- For an existing MV
ALTER MATERIALIZED VIEW my_mv SET ("resource_group" = "mv_refresh_group");

Is physical separation possible in Open Source? In the Open Source

shared_data

mode: * Physical Isolation (Warehouse): Not available. All CN nodes belong to the same default pool. * Workload Isolation (Resource Groups): Available. This is the recommended way to prevent heavy MV refreshes from impacting your dashboards or user queries. Summary of Differences Feature | Open Source | Enterprise / Cloud Shared-Data Mode | Yes | Yes CREATE WAREHOUSE | No (throws "not implemented") | Yes Resource Groups | Yes | Yes Isolation Method | Resource Group (Soft) | Warehouse (Hard/Physical) If you require strict physical separation (e.g., specific nodes that only do ingestion), you would currently need the Enterprise Edition or to use a Shared-Nothing architecture with BE Tags, though Shared-Nothing is less elastic than the Shared-Data mode you are currently using. References • administration/management/FE_configuration.md

2 Views

Open in Slack

Previous Next