Yes, you can limit the maximum CPU usage of a Compute Node (CN) in StarRocks through several methods, depending on whether you want to limit the entire process or specific workloads. 1. Process-Level Limits (Infrastructure) If you are running StarRocks on Kubernetes or in a containerized environment, the most effective way is to use infrastructure-level limits: * Kubernetes: In your

starRocksCnSpec

(if using the StarRocks Operator), you can specify

limits.cpu

.

yaml
    starRocksCnSpec:
      limits:
        cpu: 8  # Limit CN to 8 CPU cores

* Cgroups/Docker: You can set CPU quotas using Docker's

--cpus

flag or direct Linux

cgroups

to restrict the entire CN process. 2. Workload Isolation (Resource Groups) Starting from v3.3.5, StarRocks supports CPU hard limits via Resource Groups. This is the best way to ensure specific workloads don't exceed a set amount of CPU. * Exclusive CPU Cores (Hard Limit): Use the

exclusive_cpu_cores

parameter. This reserves specific cores for a group and prevents them from using more, even if other cores are idle.

sql
    CREATE RESOURCE GROUP my_limit_group
    PROPERTIES (
        "exclusive_cpu_cores" = "4", -- Hard limit of 4 cores
        "mem_limit" = "20%"
    );

* Max CPU Cores: You can also set

max_cpu_cores

to define the maximum number of cores a resource group can use on a single node. 3. Global Thresholds (Query Queues) You can set a threshold that triggers query queuing when the CN's CPU usage becomes too high. This prevents the node from becoming completely overwhelmed. * Variable:

query_queue_cpu_used_permille_limit

* Example: Setting this to

800

(80%) will cause new queries to wait in a queue if the total CPU usage on the node exceeds 80%.

sql
    SET GLOBAL query_queue_cpu_used_permille_limit = 800;

4. Configuration Tuning (CN Level) In your

cn.conf

file, you can manually override the number of CPU cores StarRocks detects. This affects the size of internal thread pools (like the pipeline engine): * Parameter:

num_cores

* If you set

num_cores = 8

on a 16-core machine, StarRocks will size its thread pools as if it only has 8 cores available. Summary of Recommendation * For hard capping the entire CN: Use Kubernetes/Container limits. * For isolating users/tenants: Use Resource Groups with

exclusive_cpu_cores

(v3.3.5+). * To prevent system overload: Use Query Queues with

query_queue_cpu_used_permille_limit

. References • faq/Others.md • administration/management/BE_configuration.md • deployment/plan_cluster.md

Based on the symptoms you're describing, it appears you have encountered a limitation in StarRocks regarding shared-data (Compute-Storage Separation) tables. The Root Cause In StarRocks, the feature to modify the bucket count or bucketing method of existing partitions using

ALTER TABLE ... DISTRIBUTED BY ... BUCKETS <num>

is currently not supported for shared-data mode. While this feature was introduced in version 3.2 for shared-nothing tables, the official documentation explicitly notes:

NOTICE: StarRocks's shared-data mode currently does not support this feature.

Why you see zero rows and errors When you run this command on a shared-data table, the metadata for the partitions can become inconsistent. The "zero rows and zero bytes" symptom suggests that the metadata was updated to reflect new bucketing information, but the actual data was not redistributed in the background (as the feature is unsupported). The

load segments error

occurs because the query engine is trying to find segment files in the object storage based on the new, incorrect bucket/tablet metadata, but those files do not exist or are not where the engine expects them to be. How to Resolve and Recommended Approach Since the metadata for those partitions is likely corrupted or points to empty sets, you will need to restore or recreate the data. 1. For existing data: The only supported way to change the bucket count for existing data in a shared-data table is to: * Create a new table with the desired bucketing configuration. _ Migrate the data using

INSERT INTO new_table SELECT _ FROM old_table;

. (Note: If the old table is completely unqueryable due to the "load segments error", you may need to restore it from a backup or re-import the data). 2. For future partitions: If you want newly created partitions to use a different bucket count without affecting existing ones, you can modify the table's default bucket count (supported in v3.2.11+, v3.3.5+, and v3.4.0+):

sql
    ALTER TABLE <table_name> MODIFY COLUMN BUCKETS <num>;
    -- Or in newer versions (v3.5.8+, v4.0.1+):
    ALTER TABLE <table_name> DISTRIBUTED BY HASH(...) DEFAULT BUCKETS <num>;

This only affects partitions created after the command is run. Recovery Hint If you need to get the table back to a working state immediately, you can try to

ALTER TABLE

back to the exact original bucket count and bucketing key. If the metadata reverts correctly, you might be able to access the original segments again, but this is not guaranteed if the metadata state has been significantly altered. References • faq/operation_maintenance_faq.md • table_design/data_distribution/Data_distribution.md • sql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md

In StarRocks, seeing 0 rows and 0 bytes for a partition in a shared-data table (especially a Primary Key table) when you know data has been loaded is usually related to how metadata is updated and reported in the shared-data architecture. Here are the most likely reasons why this is happening: 1. Primary Key Table Metadata Reporting In Primary Key tables, the

RowCount

and

DataSize

metrics shown in

SHOW PARTITIONS

or

information_schema.partitions

are often estimates that are updated asynchronously. * The Compaction Link: In shared-data mode, physical data statistics are often recalculated and pushed to the FE's metadata during compaction. If compaction has not been run in a long time (as you noted), the metadata might not have been refreshed with the latest consolidated stats from the object storage. * Asynchronous Updates: For Primary Key tables, the system prioritizes "Delete+Insert" logic. The actual "visible" row count is finalized when the BEs apply the delete vectors and report the results back to the FE. In some versions, this reporting only happens periodically or during compaction events. 2. Lack of Compaction in Shared-Data In a shared-data cluster, compaction is scheduled by the FE and performed by the Compute Nodes (CNs). * If compaction is disabled or stuck, the "Visible Version" of the partition will keep increasing, but the FE's cached statistics for that partition may remain at 0 until a successful compaction task consolidates the versions and calculates the new size/count. * You can check the compaction status and score for that partition using:

sql
    -- Replace with your DB and Table name
    SHOW PROC '/dbs/<db_id>/<table_id>/partitions';

Look for the

MaxCS

(Max Compaction Score). If it is very high, it confirms that compaction is lagging. 3. Metadata Refresh Lag The

SHOW PARTITIONS

command retrieves data from the FE's memory. In a shared-data architecture, if the FE has not received a status update from the CNs (which normally happens after load or compaction), it will show the last known state (which might be the initial 0 state). How to verify if data actually exists: Do not rely solely on

SHOW PARTITIONS

for data validation. Instead, run a direct query to see if the data is accessible:

sql
SELECT COUNT(*) FROM your_table PARTITION (your_partition_name);

If this query returns the correct number of rows, then the issue is purely a metadata display issue in the FE, likely caused by the lack of compaction you mentioned. Recommended Actions: 1. Check Compaction Status: Run

SHOW COMPACTION

to see if there are any active or failed compaction tasks. 2. Trigger Manual Compaction: If the compaction score is high and it hasn't run, you can try to trigger it manually to force a metadata update. 3. Check FE Logs: Look for

InternalTableStats

or "report" related logs in the FE to see if it's having trouble updating table statistics. 4. Use `partitions_meta`: If you are on version 3.1.9+ or 3.2.4+, check the

information_schema.partitions_meta

table for a more detailed view of the partition metadata.

sql
SELECT * FROM information_schema.partitions_meta 
WHERE TABLE_NAME = 'your_table' AND PARTITION_NAME = 'your_partition';

If the row count is critical for your operations (like for the query optimizer), running

ANALYZE TABLE <table_name>

can sometimes help force a recalculation of statistics. References • knowledge/admin_and_operation/shared_data_compaction.md