http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> does resource group queue classifie...
# questions-and-troubleshooting
r
@Rocky does resource group queue classifier have read vs write separation?
r
Yes, StarRocks resource groups support read vs. write separation through the use of the 
query_type
 classifier. By configuring this classifier, you can route read operations (SELECT) and write operations (INSERT/Load) into different resource groups, ensuring that heavy write tasks do not starve read queries of resources, or vice versa. How it works: When you create or alter a resource group, you can specify the 
query_type
in the classifier: * Read Separation: Use 
query_type in ('select')
to match query requests. * Write Separation: Use 
query_type in ('insert')
to match data loading tasks (such as 
INSERT INTO
or 
BROKER LOAD
). Example Configuration: You can create two separate resource groups—one for reads and one for writes—to isolate their CPU, memory, and concurrency limits.
Copy code
sql
-- Resource group for read operations
CREATE RESOURCE GROUP read_group
TO (query_type in ('select'))
WITH (
    "cpu_weight" = "10",
    "mem_limit" = "50%",
    "concurrency_limit" = "100"
);

-- Resource group for write operations (loading)
CREATE RESOURCE GROUP write_group
TO (query_type in ('insert'))
WITH (
    "cpu_weight" = "5",
    "mem_limit" = "30%",
    "concurrency_limit" = "20"
);
Key Points: * Classifier Matching: When a query or load task is submitted, StarRocks checks the 
query_type
. If it matches 
insert
, it follows the rules of the write group; if it's a 
select
, it follows the read group. * Queue Management: If you have enabled the query queue (
enable_group_level_query_queue = true
), the 
concurrency_limit
for each group will manage the queuing independently. This means writes can be queued separately from reads. * Supported Types: The 
query_type
classifier supports 
select
and 
insert
(available since v2.5). From v3.1.4 onwards, you can further refine these with 
plan_cpu_cost_range
and 
plan_mem_cost_range
to separate small reads from big analytical queries. References • faq/resource_isolation_faq.md • administration/management/resource_management/resource_group.md
r
@Rocky if we give mem_limit in Resource group - is this memory share not used for other resource groups? I.e is it reserved? Or when there are no items, it gets shared by other resoure group?
r
In StarRocks, the 
mem_limit
parameter in a Resource Group acts as a hard quota (upper limit) rather than a reserved block of memory. Here is the breakdown of how it works: 1. It is NOT a Reservation Setting a 
mem_limit
for a resource group does not "lock" or "pre-allocate" that memory. If a resource group is idle (has no running queries), the memory is not physically taken away from the system. It remains available for use by other processes or resource groups on the Backend (BE). 2. It is a Hard Cap While it doesn't reserve memory, it strictly limits how much memory queries assigned to that group can consume. * Even if the rest of the cluster is idle, a resource group cannot exceed its defined 
mem_limit
. * If a query in that group tries to use more memory than the 
mem_limit
allows, the query will be killed (or trigger spilling if configured). 3. Over-subscription is Allowed The sum of 
mem_limit
across all resource groups can exceed 100%. This is because StarRocks uses an over-subscription model. For example, you could have three resource groups each with a 
mem_limit
of 50%. This works as long as they don't all hit their peak memory usage at the exact same time. If the physical BE memory reaches its limit before a resource group reaches its specific 
mem_limit
, the BE's global memory management will take over (usually by failing queries to protect the node). 4. Shared Memory (v4.0+) If you want multiple resource groups to share a common pool of memory (where one group can use more if another is idle), you can use the 
mem_pool
 property (available in StarRocks 4.0 and later). By assigning multiple resource groups to the same 
mem_pool
, they will collectively draw from that pool's limit. This allows for more flexible sharing:
Copy code
sql
-- Both rg1 and rg2 share a 50% limit. 
-- If rg1 is idle, rg2 can use the full 50%.
CREATE RESOURCE GROUP rg1 WITH ("mem_limit" = "50%", "mem_pool" = "shared_pool");
CREATE RESOURCE GROUP rg2 WITH ("mem_limit" = "50%", "mem_pool" = "shared_pool");
Summary Table Feature | Behavior Is memory reserved? | No. It is only consumed when queries are running. Is it a hard limit? | Yes. A group cannot exceed its limit even if the BE is idle. Can others use idle memory? | Yes, as long as they stay within their own 
mem_limit
. Sum of limits > 100%? | Allowed. This enables over-subscription. References • faq/resource_isolation_faq.md • administration/management/resource_management/resource_group.md
r
@Rocky how to alter resource group cpu weight after creating
r
To alter the 
cpu_weight
of a resource group after it has been created, you use the 
ALTER RESOURCE GROUP
statement with the 
WITH
clause. Syntax
Copy code
sql
ALTER RESOURCE GROUP <group_name> 
WITH (
    'cpu_weight' = '<new_weight_value>'
);
Example If you have a resource group named 
analytics_rg
and you want to change its CPU weight to 
10
, run:
Copy code
sql
ALTER RESOURCE GROUP analytics_rg 
WITH (
    'cpu_weight' = '10'
);
Key Points to Remember: * Definition: 
cpu_weight
defines the relative share of CPU time allocated to a resource group when the system is under load. For example, a group with a weight of 10 gets twice as much CPU time as a group with a weight of 5. * Version Note: In versions earlier than v3.3.5, this parameter was named 
cpu_core_limit
. Starting from v3.3.5, 
cpu_weight
is the preferred term for Shared resource groups. * Restrictions: You cannot set both 
cpu_weight
and 
exclusive_cpu_cores
for the same resource group. If you want to switch from a weight-based model to a hard-limit model, you would need to adjust these parameters accordingly. * Privileges: You must have the 
ALTER
privilege on the specific resource group to perform this action. References • administration/management/resource_management/resource_group.md • sql-reference/sql-statements/cluster-management/resource_group/CREATE_RESOURCE_GROUP.md • sql-reference/sql-statements/cluster-management/resource_group/ALTER_RESOURCE_GROUP.md
r
@Rocky can i create classifier like this with multiple user with same rg/ ?
Copy code
CREATE RESOURCE GROUP rg_development
  TO (user='admin', user='development_cf', user='development_aws')
  WITH (
      'cpu_weight' = '1',
      'mem_limit' = '100%',
      'concurrency_limit' = '15',
      'type' = 'normal'
  );
r
To create a resource group for multiple users in StarRocks, you should define each user as a separate classifier within the 
TO
clause, separated by commas. The syntax you provided puts all users into a single classifier definition, which is not supported. Each classifier is a set of conditions (like 
user
, 
role
, 
source_ip
) that a query must match. If you want to include multiple users, you provide a list of classifiers. Corrected Syntax Here is how you should write your statement:
Copy code
sql
CREATE RESOURCE GROUP rg_development
  TO 
    (user='admin'), 
    (user='development_cf'), 
    (user='development_aws')
  WITH (
      'cpu_weight' = '1',
      'mem_limit' = '100%',
      'concurrency_limit' = '15',
      'type' = 'normal'
  );
Key Points: * Multiple Classifiers: Each 
(user='...')
is an independent classifier. A query will be assigned to this resource group if it matches any of the listed classifiers. * Property Matching: Properties inside a single set of parentheses (e.g., 
(user='admin', source_ip='192.168.1.1')
) follow AND logic—all conditions in that specific classifier must be met. * Resource Limits: Your 
WITH
parameters (
cpu_weight
, 
mem_limit
, etc.) will apply to the resource group as a whole, shared by all three users according to the group's internal scheduling. * Type: Using 
'type' = 'normal'
is correct for standard resource groups. If you have already created the resource group and want to add another user later, you can use the 
ALTER
command:
Copy code
sql
ALTER RESOURCE GROUP rg_development ADD (user='new_user');
References • sql-reference/sql-statements/cluster-management/resource_group/CREATE_RESOURCE_GROUP.md • administration/management/resource_management/resource_group.md
r
I used your above create query when i do show resource groups all I only see (id=737747, weight=1.0, user=development_cf)
@Rocky I used your above create query
Copy code
sql
CREATE RESOURCE GROUP rg_development
  TO 
    (user='admin'), 
    (user='development_cf'), 
    (user='development_aws')
  WITH (
      'cpu_weight' = '1',
      'mem_limit' = '100%',
      'concurrency_limit' = '15',
      'type' = 'normal'
  );
when i do show resource groups all I only see (id=737747, weight=1.0, user=development_cf)
r
It appears that when you use multiple classifiers in a single 
CREATE RESOURCE GROUP
statement, StarRocks should indeed register all of them. In the output of 
SHOW RESOURCE GROUPS ALL
, all classifiers associated with a resource group are typically displayed as a comma-separated list within the 
classifiers
column. If you are only seeing one classifier (e.g., 
(id=737747, weight=1.0, user=development_cf)
), here are a few things to check: 1. Check the specific resource group Instead of viewing all groups, try viewing the details for that specific group to see if more information is returned:
Copy code
sql
SHOW RESOURCE GROUP rg_development;
2. Use the VERBOSE keyword StarRocks supports a 
VERBOSE
flag that can sometimes reveal additional details or deprecated fields:
Copy code
sql
SHOW VERBOSE RESOURCE GROUP rg_development;
3. Verify Output Truncation If you are using a command-line client or a GUI, it is possible the output column is being truncated. You can try running the query with the 
\G
terminator in the MySQL client to see the result in a vertical format, which avoids column width issues:
Copy code
sql
SHOW RESOURCE GROUPS ALL\G
4. Manually add the missing classifiers If for some reason they weren't created, you can add them one by one using the 
ALTER RESOURCE GROUP
command:
Copy code
sql
ALTER RESOURCE GROUP rg_development ADD (user='admin');
ALTER RESOURCE GROUP rg_development ADD (user='development_aws');
5. Check for conflicts Ensure that the users 
admin
and 
development_aws
aren't already explicitly assigned to another resource group with a higher matching weight, although this usually doesn't prevent them from appearing in the classifier list of the group they were assigned to. Note: If you find that only one classifier is still showing even after an 
ALTER
, please check your StarRocks version. In some older versions, there might have been display limitations in the 
SHOW
command even if the classifiers were active. References • sql-reference/sql-statements/cluster-management/resource_group/SHOW_RESOURCE_GROUP.md • faq/resource_isolation_faq.md • administration/management/resource_management/resource_group.md • sql-reference/sql-statements/cluster-management/resource_group/CREATE_RESOURCE_GROUP.md • sql-reference/sql-statements/cluster-management/resource_group/SHOW_USAGE_RESOURCE_GROUPS.md
Open in Slack
PreviousNext
Powered by