@Rocky if some random user lanch heavy query, it would be happend OOM in fe pod. how can we prevent this problems prevously launch that?

To prevent Frontend (FE) Out of Memory (OOM) issues caused by heavy queries in StarRocks, you can implement several preventive measures ranging from JVM configuration to workload management. 1. Optimize FE JVM and Environment Settings Before queries are even launched, ensure your FE has enough "breathing room" and is configured to handle memory efficiently. * JVM Heap Size (

-Xmx

): Metadata and query planning occur in the FE heap. Ensure the heap size is consistent across all FEs and matches your scale. * < 1 million tablets: 16 GB * 1M – 2M tablets: 32 GB * Rule of thumb: Set

-Xmx

to roughly 70-80% of the total pod memory to leave room for the OS and off-heap memory. * Prevent Memory Fragmentation: Add

MALLOC_ARENA_MAX=1

to your

fe.conf

. This is a common fix for JVM-based processes in containerized environments to prevent excessive off-heap memory growth. * Offload Checkpoints: If you are on version 3.4.0+, the Leader FE can offload image checkpoints to Follower nodes, significantly reducing the Leader's memory spikes. 2. Use "Big Query" Restrictions (Resource Groups) Resource Groups are the most effective way to automatically kill "monster queries" before they crash the system. * `big_query_mem_limit`: Set a hard limit on the memory a single query can consume on a BE node. If a query exceeds this, it is terminated. * `big_query_cpu_second_limit`: Kills queries that consume too much CPU time. * `concurrency_limit`: Prevents too many heavy queries from running simultaneously, which reduces the total planning and metadata load on the FE. Example of creating a resource group to restrict big queries:

sql
CREATE RESOURCE GROUP big_query_group
      VANNELS (user='random_user')
PROPERTIES (
    "cpu_weight" = "1",
    "mem_limit" = "0.2", -- Limit group to 20% of BE mem
    "big_query_mem_limit" = "10737418240", -- 10GB limit per query
    "big_query_cpu_second_limit" = "100"   -- 100s CPU limit
);

3. Planner-Level Restrictions Complex SQL (e.g., thousands of

OR

conditions or deep joins) can exhaust FE memory during the planning phase. * `max_scalar_operator_flat_children`: (FE Config) Default is 10,000. Reducing this can prevent the optimizer from using too much memory on massive expression trees. * `max_table_partition_num`: Limits the number of partitions a single query can access. This prevents "select all" queries on tables with millions of partitions from overloading the FE. 4. Session Variables for Users You can set session variables globally or per user to act as a "circuit breaker": * `query_timeout`: Set a reasonable timeout (e.g., 300s) so queries don't hang and hold memory for hours. * `query_mem_limit`: Limits the memory used by a query on each BE. * `max_scan_key_num`: Limits the number of scan keys generated by the planner, preventing memory-intensive filter planning. 5. Proactive Monitoring If you suspect a query is currently running that might cause an OOM: * Real-time Check: Run

SHOW PROC '/current_queries';

to see

MemoryUsageBytes

and

CPUCostSeconds

for active queries. * Memory Profiling (v3.3.6+): StarRocks now automatically prints memory profiles in

fe/log/proc_profile

. You can look at the flame graphs there to identify which part of the FE (e.g., Global Dictionary, Planner, or Metadata) is consuming the most memory. Summary Recommendation: Start by setting a Resource Group for your ad-hoc users with a strict

big_query_mem_limit

. Then, ensure your FE JVM is correctly sized for your tablet count and use

MALLOC_ARENA_MAX=1

to stabilize memory usage. References • knowledge/trouble_shooting/fe_memory_problems.md • faq/Sql_faq.md • knowledge/trouble_shooting/fe_deadlock_problems.md

You can create roles, then attach roles to the users. You need to then grant some permission to the roles, such as only querying specific tables (prevent unnecesarily querying really large tables) or allowing only certain types of query (delete/insert/select/etc.). Another way, you can use sql blacklist. https://docs.starrocks.io/docs/sql-reference/sql-statements/cluster-management/sql_blacklist/ADD_SQLBLACKLIST/ https://docs.starrocks.io/docs/sql-reference/sql-statements/account-management/GRANT/