http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> Is there a way I can alter my prima...
# questions-and-troubleshooting
m
@Rocky Is there a way I can alter my primary key table to use a fixed bucket size for it's distribution key after creation? I now have a use case for colocation joins and need to ensure the distribution key and bucket for both tables in the colocation group is the same.
r
Yes, you can alter an existing Primary Key table to use a fixed bucket size (number of buckets) and a specific distribution key. Starting from StarRocks v3.2, you no longer need to recreate the table to change its distribution strategy. To achieve your goal of setting up a Colocate Join, follow these steps: 1. Modify the Bucketing Scheme Use the 
ALTER TABLE
statement to specify the new distribution key and the exact number of buckets. For a Colocate Join, the distribution columns' data types, order, and bucket count must be identical across all tables in the group.
Copy code
sql
-- Example: Changing the table to have 16 buckets distributed by 'user_id'
ALTER TABLE your_pk_table 
DISTRIBUTED BY HASH(user_id) BUCKETS 16;
Important Constraints for Primary Key Tables: * PK Subset: The distribution columns you choose must be a subset of the Primary Key columns. * Asynchronous Task: This is a schema change operation. You can monitor its progress using:
Copy code
sql
    SHOW ALTER TABLE COLUMN;
* Cluster Mode: This feature is currently supported in shared-storage (standard) mode but is not supported in the cloud-native shared-data mode. 2. Join the Colocation Group Once the redistribution task is complete, you can add the table to a colocation group. Ensure the other table is also in the same group and shares the same configuration.
Copy code
sql
-- Step 2: Set the colocation group property
ALTER TABLE your_pk_table 
SET ("colocate_with" = "my_colocation_group");
Key Requirements for Colocate Joins To ensure the 
colocate_with
property is accepted and the join works efficiently: 1. Distribution Key: The types and order of columns in 
DISTRIBUTED BY HASH(...)
must match exactly between the tables. 2. Bucket Count: Both tables must have the exact same number of buckets. 3. Replication Num: The tables should generally have the same number of replicas (e.g., 
3
). 4. Status Check: You can verify if the colocation group is healthy and balanced by running:
Copy code
sql
    SHOW PROC '/colocation_group';
Summary of "Fixed Bucket Size" If you previously used StarRocks' Automatic Bucketing (where the system determines the bucket count), the 
ALTER TABLE ... BUCKETS <num>
command effectively "pins" the table to a fixed count, which is a prerequisite for Colocate Join since the system cannot dynamically change bucket counts for tables within a colocation group. References • sql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md • best_practices/query_tuning/schema_tuning.md • using_starrocks/Colocate_join.md • best_practices/bucketing.md • sql-reference/sql-statements/table_bucket_part_index/CREATE_TABLE.md
m
@Rocky How can I inspect the current bucketing of my primary key table to the new bucket value I choose does not impact query/ingestion performance.
r
To inspect your current bucketing and determine an optimal fixed bucket size for your Primary Key table, follow these steps to ensure you maintain high query and ingestion performance. 1. Inspect Current Bucketing and Data Volume You need to know how much data is currently in your partitions and how many buckets (tablets) are presently being used. * Option A: Using SQL (Recommended for v3.2+) Query the 
information_schema
to get a clear overview of data size and bucket counts across all partitions:
Copy code
sql
    SELECT
        PARTITION_NAME,
        BUCKETS,
        DATA_SIZE,
        ROW_COUNT
    FROM information_schema.partitions_meta
    WHERE TABLE_NAME = 'your_table_name'
    ORDER BY DATA_SIZE DESC;
* Option B: Using Administrative Commands
Copy code
sql
    SHOW PARTITIONS FROM your_table_name;
Look for the 
Buckets
and 
DataSize
columns. 2. Check for Data Skew Before changing the distribution key, ensure the columns you choose will distribute data evenly. If data is skewed (one bucket is much larger than others), ingestion and queries will be throttled by the slowest node.
Copy code
sql
-- Check if tablets are evenly distributed across BE nodes
ADMIN SHOW REPLICA DISTRIBUTION FROM your_table_name;
If you see a significant imbalance in "ReplicaNum" or size across BEs, your current distribution key might be causing skew. Ensure your new distribution key has high cardinality (many unique values). 3. Choosing the New Bucket Value When choosing a fixed bucket count for a Colocation Group, aim for a "sweet spot" based on data volume and cluster size: * Target Tablet Size: Aim for 1 GB to 10 GB of compressed data per tablet (bucket). _ _Too few buckets:* Limits query parallelism and can make compaction slow. _ _Too many buckets:* Increases metadata overhead on the FE and can slow down small queries. * Parallelism Rule of Thumb: Ensure the total number of buckets is at least equal to the number of BE nodes (or a multiple like 2x) to utilize all CPU resources. _ _Example:* If you have 3 BEs and 30 GB of data per partition, 6 or 9 buckets is a good starting point (~3.3 GB to 5 GB per tablet). * Automatic Formula Reference: If you are unsure, StarRocks’ internal logic often follows this pattern for cluster resource utilization (where X = Number of BEs): _ If X < 12: 
2 _ X
buckets. _ If 12 <= X <= 24: 
1.5 _ X
buckets. 4. Performance Considerations for Colocation Since you are altering this for a Colocate Join, remember: 1. Consistency is Key: Both tables in the 
colocate_with
group must have the exact same number of buckets. If Table A needs 64 buckets for its data volume, Table B must also have 64, even if Table B is much smaller. 2. PK Constraint: For Primary Key tables, your distribution key must be a subset of the Primary Key. 3. Monitoring the Change: Altering the bucket count is an asynchronous process. Monitor it with:
Copy code
sql
    SHOW ALTER TABLE COLUMN;
Summary Checklist Goal | Action Check Current Size | Use 
information_schema.partitions_meta
. Choose Key | Use a high-cardinality subset of your PK. Set Buckets | Total Data Size / (1 to 10 GB) = Target Bucket Count. Verify Skew | Use 
ADMIN SHOW REPLICA DISTRIBUTION
. References • knowledge/usage/best-practice.md • table_design/data_distribution/Data_distribution.md
m
@Rocky Is tablet equivalent to bucket? The query shows 100-110 buckets per partition, so I guess 128 buckets will suffice?
r
Yes, a tablet is the physical equivalent of a bucket. When you define a distribution with 
BUCKETS 128
, StarRocks logically divides your partition into 128 buckets. Physically, each of these buckets is stored as a tablet (which consists of the actual data files and replicas across your BE nodes). Regarding your choice of 128 buckets, here is how to determine if that is the right "fixed" number for your Colocation Group: 1. Is 128 a good number? If your table currently has 100–110 buckets per partition and is performing well, 128 is a very reasonable choice. * Parallelism: Using a power of 2 (like 128) is a common convention in distributed systems, though not strictly required by StarRocks. * Comparison: It is close to your current count, so you likely won't see a significant change in performance overhead. 2. The "Sanity Check" Calculation Before you commit to 128 for a colocation group (since all tables in the group must then use 128), run these two checks: A. Data Size Check (Target: 100 MB – 10 GB per tablet) Check your largest partition's data size and divide by 128: * Formula: 
Max Partition Size / 128
* Result < 100 MB: You might have too many buckets (high metadata overhead). * Result > 10 GB: You might have too few (slow scans and compaction issues). _ _If 128 puts you in the 500 MB to 5 GB range, it is perfect.* B. Metadata Check (Target: < 200,000 tablets per BE) Since every partition has its own set of buckets, the total tablet count can grow quickly. * Formula: 
(Total Partitions _ 128 buckets _ Replication Factor) / Number of BEs
* If this number exceeds 200,000 per BE, you may see FE (Front-End) memory pressure and slower metadata operations. 3. Colocation Reminder Since your goal is a Colocate Join, once you alter Table A to 128 buckets, you must also alter Table B (and any others in the group) to 128 buckets. * The Distribution Columns must also match in data type and order. * The Replication Number should ideally be the same. How to execute the change:
Copy code
sql
-- 1. Change bucket count to a fixed 128
ALTER TABLE your_table 
DISTRIBUTED BY HASH(join_key_column) BUCKETS 128;

-- 2. Monitor progress (it takes time to redistribute data)
SHOW ALTER TABLE COLUMN;

-- 3. Once finished, link to the colocation group
ALTER TABLE your_table 
SET ("colocate_with" = "your_group_name");
Rocky's Pro Tip: If you have many very small partitions (e.g., hourly partitions with only a few MBs of data), 128 buckets per partition might be overkill. In that case, consider merging partitions (e.g., daily) or reducing the bucket count if the total metadata count becomes a becomes a concern. Otherwise, 128 is a solid, safe choice for most medium-to-large production workloads. References • best_practices/bucketing.md • knowledge/usage/best-practice.md • best_practices/query_tuning/schema_tuning.md • table_design/data_distribution/Data_distribution.md
m
@Rocky Different question. I'm trying to set up resource groups. I'm thinking default_wg has default cpu weight 16. High priority rg gets 3 weight and low priority gets 1 weight. This guarnatees 80% cpu to ingestion for burst activity. I want my user 
'sdm_reader'@'%'
to get low priority. Should I just let the cpu weight throttle the mem_limit? What limitations should I configure in each resource groups? The high and low priorities are mainly for queries. Give me step by step guide
r
Setting up Resource Groups (also known as Worker Groups) is the right way to achieve this. Your plan to use a 1631 ratio (80%/15%/5%) for CPU weights is mathematically sound for proportional sharing. However, there is a critical distinction in how StarRocks handles resources: CPU weight is a soft limit, while Memory limit is a hard percentage of the query pool. 1. CPU vs. Memory: The "Throttling" Logic * CPU (
cpu_weight
): This is a proportional share. If the system is under full load, resources are divided by weight (16/20, 3/20, etc.). If the system is otherwise idle, a "Low Priority" group can use 100% of the CPU. It is "soft" because it doesn't waste cycles if they are available. * Memory (
mem_limit
): This is a hard limit (expressed as a percentage of the BE's total query memory pool). If you set a 
mem_limit
of 5% for the low-priority group, that user will be killed (OOM) if they exceed 5%, even if the rest of the cluster is 99% idle. Recommendation: Do not rely on CPU weight to throttle memory. You should set specific memory limits to prevent a "Low Priority" user from accidentally crashing a Backend node by consuming all available RAM. ────────── Step-by-Step Guide Step 1: Adjust the 
default_wg
(Ingestion) By default, all loads (Stream Load, Routine Load, etc.) and unclassified queries fall into 
default_wg
. We will pin its weight to 16 to ensure your 80% goal.
Copy code
sql
-- Set the default weight. Note: In some versions < 3.3, use 'cpu_core_limit' instead of 'cpu_weight'
ALTER RESOURCE GROUP default_wg WITH (
    'cpu_weight' = '16',
    'mem_limit' = '80%'
);
Step 2: Create the High Priority Group This group is for your critical queries. We'll give it 15% of the shares (3/20).
Copy code
sql
CREATE RESOURCE GROUP high_priority_rg 
WITH (
    'cpu_weight' = '3',
    'mem_limit' = '15%',
    'concurrency_limit' = '20' -- Prevents too many high-priority queries from competing
);
Note: You can add classifiers to this group later based on specific roles or databases. Step 3: Create the Low Priority Group for 
sdm_reader
 We will assign the user 
'sdm_reader'
here. We'll give it the lowest CPU weight and a strict memory/query limit to protect the cluster.
Copy code
sql
CREATE RESOURCE GROUP low_priority_rg 
TO (user='sdm_reader') -- This is the classifier
WITH (
    'cpu_weight' = '1',
    'mem_limit' = '5%',
    'concurrency_limit' = '5', -- Limit simultaneous low-priority scans
    'big_query_mem_limit' = '2147483648', -- Kill any single query in this group over 2GB
    'big_query_cpu_second_limit' = '300'  -- Kill any query in this group taking > 5 mins CPU time
);
Step 4: Verify the Configuration Check that the weights and classifiers are applied correctly:
Copy code
sql
-- View all groups and their ratios
SHOW RESOURCE GROUPS ALL;

-- Check which group a query is running in (useful for testing sdm_reader)
-- Run a query as sdm_reader, then check:
SELECT * FROM information_schema.active_queries;
────────── Recommended Limitations for Each Group Parameter | Default (Ingestion) | High Priority | Low Priority | Reason 
cpu_weight
 | 16 | 3 | 1 | Ensures 80/15/5 ratio during CPU contention. 
mem_limit
 | 80% | 15% | 5% | Prevents low-priority queries from starving ingestion of RAM. 
concurrency_limit
 | 0 (unlimited) | 20-50 | 5-10 | Prevents "connection storms" from overwhelming the BE. 
big_query_mem_limit
 | 0 | 0 (or high) | 2GB - 5GB | Critical: Ensures one bad scan by 
sdm_reader
doesn't hit the 5% limit and trigger a group-wide failure. Important Tips: 1. Ingestion User: Ensure your ingestion process (e.g., Flink, Routine Load) is not running as 
sdm_reader
, otherwise it will be throttled to 5% CPU/Memory. 2. User Matching: The classifier 
user='sdm_reader'
matches the username. StarRocks resource groups do not currently distinguish between 
'user'@'host1'
and `'user'@'host2'`; it matches the base username.
3. The "Burst" Logic: Because CPU weight is a soft share, if no queries are running in the High/Low groups, Ingestion will automatically use 100% of the CPU. The 80% guarantee only kicks in the moment a query starts in the other groups. References • administration/management/resource_management/resource_group.md • faq/resource_isolation_faq.md • sql-reference/sql-statements/cluster-management/resource_group/CREATE_RESOURCE_GROUP.md • administration/management/resource_management/query_queues.md • administration/management/resource_management/Resource_isolation.md References • administration/management/resource_management/resource_group.md • faq/resource_isolation_faq.md
m
@Rocky No shared memory pool? So each group reserves its own memory?
r
That is a great observation. The way StarRocks handles memory in resource groups is often misunderstood: It is a quota (limit), not a reservation. Here is the breakdown of how the memory "pool" actually works: 1. No Pre-allocation (No Reservation) When you set 
'mem_limit' = '15%'
, StarRocks does not sit there and keep 15% of the RAM empty just for that group. * If the High Priority group is idle, the Ingestion group (
default_wg
) can use all available memory (up to its own limit). * Memory is only "claimed" when a query actually runs. 2. The Global BE Query Pool All resource groups draw from the same physical memory pool on the Backend (BE), which is typically 90% of the BE's total RAM (controlled by 
mem_limit
in 
be.conf
). * The Over-subscription Rule: You can actually set the sum of all resource group 
mem_limit
values to more than 100%. _ _Example:* You could give Group A 80%, Group B 40%, and Group C 20%. * StarRocks allows this because it assumes not every group will hit its peak at the exact same millisecond. * However, if the physical BE RAM fills up, the system will start killing queries based on which group is exceeding its quota or which individual query is the largest. 3. Shared Memory Pools (v4.0+) If you are on StarRocks v4.0 or higher, there is a new parameter called 
mem_pool
. * If you don't specify it, every group is independent. * If you specify the same 
mem_pool
name for two groups, they share a single memory quota. This is useful if you have "Marketing" and "Finance" users who you want to collectively limit to 20% total, regardless of which department is more active. 4. Memory is a "Hard" Limit Unlike CPU weight (which is "soft" and allows groups to borrow idle cycles), Memory is a "Hard" limit. * If your Low Priority user (
sdm_reader
) hits that 5% limit, their query will fail with an "Out of Memory" error, even if 90% of the server's RAM is currently free. * This is why I recommended setting a 
big_query_mem_limit
for that user—it's better to kill one massive query than to have the entire group hit its 5% ceiling and stop all small queries for that user. Comparison: Memory vs. CPU Feature | CPU Weight | Memory Limit | Exclusive CPU Cores Type | Soft Share | Hard Quota | Hard Reservation Idle Usage | Can borrow from others | Can borrow from others* | Cannot be borrowed Limit Enforcement | Only when congested | Always enforced | Always enforced Sum of parts | Can be anything | Can exceed 100% | Must be < Total Cores _\_Note: While a group can use idle memory, it can never exceed its own 
mem_limit
percentage.* How this affects your plan: Your plan (1631 weight) is perfect for CPU. For memory, since you want to guarantee 80% to ingestion, setting the others to 15% and 5% is the safest way to ensure that even if a user writes a "query from hell," they physically cannot take enough RAM to starve your ingestion process. Rocky's Advice: Start with these hard limits. If you find your 
sdm_reader
is getting "Memory Limit Exceeded" errors too often while the cluster is idle, you can safely increase their 
mem_limit
to 10% or 20%, because the CPU weight (16:1) will still ensure the ingestion gets the actual processing priority. References • administration/management/resource_management/resource_group.md • faq/resource_isolation_faq.md • administration/management/resource_management/Resource_isolation.md References • administration/management/resource_management/resource_group.md
m
@Rocky How would mem_pool work? Is it the same idea of mem_limit, where I put two resource groups in same mem_pool and specify mem_limit for that pool?
r
You are exactly right. The 
mem_pool
(introduced in StarRocks v4.0) allows you to group multiple resource groups so they share a single memory "bucket." Here is how it works and how to set it up based on your logic: How 
mem_pool
Works 1. Shared Quota: If Group A and Group B are in the same 
mem_pool
, their combined memory usage cannot exceed the 
mem_limit
. 2. Identical 
mem_limit
Requirement: When using a pool, all resource groups in that pool must be configured with the same 
mem_limit
value. This value represents the total limit for the entire pool, not the limit for each individual group. 3. Independence: Even though they share memory, they still keep their own 
cpu_weight
, 
concurrency_limit
, and 
classifiers
. Step-by-Step Configuration If you want your High Priority and Low Priority queries to collectively never take more than 20% of the RAM (leaving 80% guaranteed for ingestion), you would configure them like this:
Copy code
sql
-- 1. Create High Priority Group in the "query_pool"
CREATE RESOURCE GROUP high_priority_rg 
WITH (
    'cpu_weight' = '3',
    'mem_limit' = '20%',        -- The total limit for the POOL
    'mem_pool' = 'query_pool',  -- The pool name
    'concurrency_limit' = '20'
);

-- 2. Create Low Priority Group in the SAME "query_pool"
CREATE RESOURCE GROUP low_priority_rg 
TO (user='sdm_reader') 
WITH (
    'cpu_weight' = '1',
    'mem_limit' = '20%',        -- Must match the pool's limit exactly
    'mem_pool' = 'query_pool',  -- Joins the same pool
    'concurrency_limit' = '5'
);
Why would you do this? * Flexible Distribution: If your 
sdm_reader
is idle, the High Priority group can use the full 20% of the memory. * Hard Cap: If both are running heavy queries, they will fight for that 20% but will never encroach on the 80% you've allocated to the 
default_wg
(ingestion). Comparison: With vs. Without 
mem_pool
 Feature | Individual Limits (Your first plan) | Shared 
mem_pool
(v4.0+) High Priority Limit | 15% | Combined 20% Low Priority Limit | 5% | Combined 20% The "Stealing" Logic | Even if Low is idle, High is stuck at 15%. | If Low is idle, High can take all 20%. Configuration | Different 
mem_limit
per group. | Must have identical 
mem_limit
. Rocky's Recommendation If you are on v4.0 or newer, use the 
mem_pool
approach. It is much more efficient because it allows your high-priority users to "borrow" the low-priority user's memory quota when they aren't using it, while still keeping a strict firewall around your ingestion (
default_wg
) memory. One final check: Run 
SHOW RESOURCE GROUPS ALL;
after creating them to ensure the 
mem_pool
column shows they are linked correctly! References • administration/management/resource_management/resource_group.md • faq/resource_isolation_faq.md • administration/management/resource_management/Resource_isolation.md • administration/management/monitoring/metrics.md References • administration/management/resource_management/resource_group.md • administration/management/monitoring/metrics.md • faq/resource_isolation_faq.md
m
@Rocky
Copy code
If FE and BE nodes are both healthy (SHOW BACKENDS indicate all BE nodes alive), it was most likely caused by insufficient disk. StarRocks has a variable in information_schema.be_configs - storage_flood_stage_left_capacity_bytes which defines the disk percentage threshold that the BE nodes enter flood state. The default is 95%, after which it will start rejecting all incoming requests.

Disk diagnosis:

accessbot starrocks-core-services-be-<n>.us-east-1a

ncdu /mnt/starrocks/data[1/2/3/4]/be/storage (We have 4 EBS volumes per node data1-4). Verify the trash directory is not occupying majority of the space. By default StarRocks keeps trash for a day but we have re-configured this to 1 hour.

Tablet diagnosis:

Ensure you have starrocks-core-services-maintain sdm resource. You need root role (SET role 'root') to perform the diagnosis below.

SHOW PROC '/statistic'

Check column UnhealthyTabletNum for the DbName of concern.

If it is more than 0, SHOW PROC '/statistics/<DbId>. The column UnhealthyTablets shows the IDs of tablets which are unhealthy.

SHOW TABLET <TabletId> - Copy and run the command in column DetailCmd. It will then show you which partition this tablet belongs to.

ALTER TABLE <table> DROP PARTITION <partition_name>

Run SHOW PROC '/statistic again and verify the number of unhealthy tablets has decreased.

Repeat for the remaining tablets. This should remediate the issue.
This is my runbook for solving stream load pipe error. Happened once due to disk saturation and tablets were unhealthy. Is there a better lesser impact remediation?
r
Dropping partitions is a "nuclear option" because it permanently deletes data. While it fixes the issue by freeing up space and removing the problematic tablets, there are several lower-impact steps you can take to remediate the situation without data loss. 1. Emergency: Raise the Flood Stage Threshold If you are at 95.1% and StarRocks has stopped accepting loads, you can temporarily "buy time" by raising the threshold to 98%. This allows the system to continue running while you perform cleanups or expand disks. Run on FE:
Copy code
sql
-- Raise the percentage limit to 98%
ADMIN SET FRONTEND CONFIG ("storage_usage_hard_limit_percent" = "98");

-- If the absolute byte limit is also blocking you, lower it (e.g., to 10GB)
ADMIN SET FRONTEND CONFIG ("storage_usage_hard_limit_reserve_bytes" = "10737418240");
Note: You must also update the BE configs (
be.conf
) or use 
curl
to update them live if you want this to persist. 2. Force Tablet Repair (Instead of Dropping) Unhealthy tablets usually occur because one replica is "stale" or "bad" due to the disk being full. Instead of dropping the whole partition, tell StarRocks to prioritize repairing the specific table.
Copy code
sql
-- This tells the FE to schedule repair tasks for this table with VERY_HIGH priority
ADMIN REPAIR TABLE <db_name>.<table_name>;

-- If only specific partitions are unhealthy:
ADMIN REPAIR TABLE <db_name>.<table_name> PARTITION (p1, p2);
Check progress with: 
ADMIN SHOW REPLICA STATUS FROM <table_name>;
3. Targeted Replica "Bad" Marking If a specific BE is corrupted or its disk is full, you can tell StarRocks that the replica on that specific BE is bad. StarRocks will then automatically try to rebuild that replica on a different, healthy BE node that has space.
Copy code
sql
-- Find the TabletId and BackendId from your SHOW TABLET command
ADMIN SET REPLICA STATUS PROPERTIES("tablet_id" = "12345", "backend_id" = "10001", "status" = "bad");
4. Trigger Immediate Trash Cleanup You mentioned the trash is kept for 1 hour. You can manually force the BE to sweep the trash and deleted bits immediately to reclaim every possible byte. Run on each BE node:
Copy code
bash
curl -X POST http://<be_ip>:<be_http_port>/api/bootstrap?check_tablet_after_start=true
# Or trigger a manual compaction run to merge small files and delete 'tombstones'
curl -X POST http://<be_ip>:<be_http_port>/api/compaction/run?tablet_id=<tablet_id>&compact_type=cumulative
5. Expand EBS Volumes (Zero Downtime) Since your runbook mentions 
us-east-1a
and EBS volumes, the cleanest way to fix "Disk Saturation" is to expand the volumes in the AWS Console. 1. Increase the EBS volume size in AWS. 2. Log into the BE and run 
sudo resize2fs
(or 
xfs_growfs
). 3. StarRocks will automatically detect the new 
capacity
without a restart. This is the lowest impact remediation as it involves zero data loss and zero downtime. Revised "Low Impact" Workflow Instead of dropping partitions immediately, follow this order: 1. Threshold Adjust: Raise 
storage_usage_hard_limit_percent
to 98% (Emergency relief). 2. Infrastructure: Expand EBS volumes and grow the filesystem (Permanent fix). 3. Repair: Run 
ADMIN REPAIR TABLE
for the tables with unhealthy tablets. 4. Wait: Give the "Tablet Scheduler" 5–10 minutes to re-replicate data to the new space. 5. Clean: If space is still tight, identify Old/Cold partitions and use 
ALTER TABLE ... SET ("storage_medium" = "HDD")
or move them to S3 (if using Shared-Data) rather than dropping them. Summary of Limits to Monitor Parameter | Default | Action in Crisis 
storage_usage_soft_limit_percent
| 90% | Raise to 95% (Allows clones/balancing to start again) 
storage_usage_hard_limit_percent
| 95% | Raise to 98% (Allows Loads/Ingestion to start again) 
trash_file_expire_time_sec
| 86400 (24h) | You have it at 3600 (1h)—this is already optimal. References
• administration/management/FE_configuration.md • sql-reference/sql-statements/cluster-management/tablet_replica/ADMIN_REPAIR.md • administration/management/BE_configuration.md • administration/management/resource_management/Replica.md References • administration/management/FE_configuration.md • sql-reference/sql-statements/cluster-management/tablet_replica/ADMIN_REPAIR.md • administration/management/BE_configuration.md • administration/management/resource_management/Replica.md • faq/operation_maintenance_faq.md References • sql-reference/sql-statements/cluster-management/tablet_replica/ADMIN_REPAIR.md • administration/management/resource_management/Replica.md • sql-reference/sql-statements/cluster-management/tablet_replica/ADMIN_CHECK_TABLET.md • sql-reference/sql-statements/cluster-management/tablet_replica/ADMIN_CANCEL_REPAIR.md • sql-reference/sql-statements/cluster-management/tablet_replica/ADMIN_SET_REPLICA_STATUS.md
Open in Slack
PreviousNext
Powered by