http://starrocks.io logo
Join Slack
Powered by
Hello Team/<@U0A71G31CDV>, we are trying to add a ...
# questions-and-troubleshooting
m
Hello Team/@Rocky, we are trying to add a metrics based on how much data files being scaned by using below query however it does not show (meaning '0' always so is there any alternative) please help us. thanks 
*select* LABEL,
STATE,
SCAN_BYTES *FROM* information_schema.loads
SCAN_BYTES is always 0 for BROKER LOAD so
@Rocky  please help me on this. thanks
r
For Broker Load jobs in StarRocks (v3.1 and later), it is a known behavior that the 
SCAN_BYTES
column in 
information_schema.loads
may return 
0
. Instead, the scanned data metrics for Broker Load are stored within the 
RUNTIME_DETAILS
 column as a JSON string. You can extract the total size of the files read using the 
get_json_object
function. Recommended Alternative Query Use the following query to extract the scanned file size (in bytes) and the number of files:
Copy code
sql
SELECT 
    LABEL,
    STATE,
    get_json_object(RUNTIME_DETAILS, '$.file_size') AS SCAN_BYTES_ALT,
    get_json_object(RUNTIME_DETAILS, '$.file_num') AS FILE_COUNT
FROM information_schema.loads
WHERE TYPE = 'BROKER';
Why 
SCAN_BYTES
is 0 In the 
information_schema.loads
view: * 
SCAN_BYTES
: Often tracks internal execution engine scan metrics, which may not be fully populated for the asynchronous Broker Load process in all versions. * 
RUNTIME_DETAILS
: Contains job-specific metadata. For Broker Load, this includes 
file_size
(total bytes of source files) and 
file_num
(total number of files), which are the authoritative metrics for data volume processed. Other Options 1. SHOW LOAD: You can still use the traditional command to see these details in the 
JobDetails
column:
Copy code
sql
    SHOW LOAD WHERE LABEL = 'your_label';
2. Query Profile: For granular performance analysis (e.g., if you want to see per-node scan rates), you can use the 
PROFILE_ID
from the 
loads
table:
Copy code
sql
    ANALYZE PROFILE FROM 'your_profile_id';
If you are on a version earlier than 3.1, 
information_schema.loads
is not available, and you should rely solely on 
SHOW LOAD
. References • administration/management/monitoring/metrics.md • administration/management/FE_configuration.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md
m
@Rocky Are labels unique globally or only within a database with regards to in the starrocks broker load with upsert/delete operation LABEL <labelName>. help me understand
r
In StarRocks, labels for Broker Load (and other load methods) are unique within a database, not globally across the entire cluster. Key Points on Label Uniqueness: * Database Scope: You can use the same label name in two different databases (e.g., 
db1.label_001
and 
db2.label_001
are allowed). However, within a single database, the label must be unique. * Preventing Duplicates: The primary purpose of a label is to prevent the same data from being loaded multiple times (Exactly-Once semantics). If you attempt to start a new load with a label that already exists in that database and has finished successfully, the system will reject it. * Reuse Policy: * FINISHED: If a load job succeeds, the label cannot be reused. * CANCELLED: If a load job fails or is cancelled, the label can be reused to retry the operation. * Retention: By default, StarRocks keeps load labels in the system for 3 days (configurable via 
label_keep_max_second
in the FE configuration). After this period, the label is cleaned up and can be reused. Usage in Upsert/Delete When performing Upsert or Delete operations via Broker Load (typically by setting the 
__op
column or using the 
MERGE ON
/ 
PRIMARY KEY
model), the label rules remain the same. The label identifies the transactional attempt to modify the data. Recommendation For automated pipelines, it is best practice to include a timestamp or a unique identifier in your label name to avoid collisions, for example: 
LOAD LABEL my_db.upsert_orders_20231027_001 ...
References • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • loading/loading_introduction/loading_concepts.md References • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • loading/loading_introduction/loading_concepts.md
m
if same label being created by 2 databases then information_schema.loads will return 2 record without using database in the partition key? is that true @Rocky
r
Yes, that is correct. Since labels are unique only within a database, if you use the same label name in two different databases, 
information_schema.loads
will return two separate records (assuming both jobs were initiated and are still within the retention period). Here is how the data would look in `information_schema.loads`: ID | LABEL | DB_NAME | STATE 10001 | 
my_label
| 
db_finance
| FINISHED 10002 | 
my_label
| 
db_sales
| FINISHED Key Details: 1. Multiple Records: Because 
information_schema.loads
is a global view of all load history (that your user has permission to see), it does not automatically filter by your current database context unless you add a 
WHERE
clause. 2. Distinguishing Jobs: You should always include 
DB_NAME
 in your query if you are looking for a specific load job to avoid ambiguity:
Copy code
sql
    SELECT *
    FROM information_schema.loads
    WHERE LABEL = 'your_label_name'
      AND DB_NAME = 'your_db_name';
3. Retention: Remember that records only stay in this view for as long as the 
label_keep_max_second
configuration allows (default is 3 days). After that, the records are purged and the label can be reused even within the same database. References • sql-reference/information_schema/loads.md • loading/loading_introduction/loading_concepts.md References • sql-reference/information_schema/loads.md • sql-reference/information_schema/stream_loads.md
m
@Rocky when we run broker load job and also, we get "_statistics_" database info also so help me what is this database and is this related to real broker load job info.
r
The 
_statistics_
 database is an internal system database in StarRocks used by the Cost-Based Optimizer (CBO). It is not a place where your Broker Load job logs are stored, but it is closely related to what happens immediately after a load finishes. Here is the breakdown of what it is and why you are seeing it: 1. What is the 
_statistics_
database? It stores metadata about the data distribution in your tables, such as: * Row counts and Data sizes. * NDV (Number of Distinct Values) for columns. * Min/Max values and Histograms. The CBO uses this information to determine the most efficient way to run your SQL queries (e.g., choosing join orders or indexes). 2. Why does it appear when you run a Broker Load? StarRocks has a feature called Automatic Statistics Collection. * When a Broker Load (or any load) finishes, the data in your table changes. * The system monitors these changes. If the amount of new data exceeds a certain threshold (by default, if more than 20% of the data has changed), StarRocks automatically triggers an internal 
ANALYZE
 job. * This job scans the table and writes the new statistical metadata into the 
_statistics_
database. 3. Is it related to "Real Broker Load Job Info"? No. The 
_statistics_
database does not contain the "status" or "metrics" of your Broker Load job (such as labels, error messages, or bytes scanned). * For Load Metrics: Use 
information_schema.loads
or 
SHOW LOAD
. * For Table Optimization: The 
_statistics_
database ensures that after your Broker Load adds millions of rows, your queries on that table remain fast because the optimizer knows exactly how much data was added. How to check these background tasks If you see activity related to this database and want to see what is happening, you can run:
Copy code
sql
-- See background statistics collection tasks
SHOW ANALYZE JOB;

-- See the status of the internal statistics tables
SHOW PROC '/statistic';
Summary: You can safely ignore the 
_statistics_
database. It is a sign that StarRocks is automatically optimizing your tables for performance after your data has been loaded. References • using_starrocks/Cost_based_optimizer.md • sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_PROC.md • sql-reference/information_schema/statistics.md schema/statistics) References • using_starrocks/Cost_based_optimizer.md
m
@Rocky help me understand these questions please, we r running into OOM issues when huge data load run via broker load so Some questions worth investigating: • Why does a Broker Load become concentrated on a single CN? • Can Broker Load execution be distributed more evenly across CN nodes? • Can we estimate incoming S3 file sizes and proactively control memory allocation? • Are there StarRocks configuration options that better limit memory consumption during large Broker Loads? • Can Broker Load memory be bounded more effectively?
r
To help you resolve OOM (Out of Memory) issues during large Broker Loads on CN (Compute Node) clusters, here is a breakdown of how StarRocks handles distribution and memory management. 1. Why does a Broker Load become concentrated on a single CN? A Broker Load is split into tasks, and tasks are split into instances. Concentration usually happens for these reasons: * Non-splittable formats: If your source files are compressed with GZIP or BZIP2, they cannot be split. StarRocks must assign the entire file to a single CN to decompress and read it. * Small file count: If you are loading a single large file (even if it's splittable like Parquet or plain Text), the number of instances is limited by the formula: 
min(Data Size / min_bytes_per_broker_scanner, Number of CNs)
. * High 
min_bytes_per_broker_scanner
: If this FE configuration is too high (default 64MB), and your total data size is small relative to that value, StarRocks may only create one or two instances. 2. Can Broker Load execution be distributed more evenly? Yes. To force higher parallelism across your CN nodes: * Adjust FE Configurations: * Decrease `min_bytes_per_broker_scanner`: This creates more instances for the same amount of data. * Increase `load_parallel_instance_num`: This allows each CN to run more parallel threads for a single load job. * File Splitting: Ensure your S3 data consists of multiple smaller files (e.g., 100MB–1GB each) rather than one massive file. * Use Splittable Formats: Use Parquet, ORC, or uncompressed Text/CSV. These formats allow multiple CNs to read different "offsets" of the same file simultaneously. 3. Can we estimate file sizes and control memory proactively? StarRocks does not have a "dry-run" memory estimator, but you can control it via: * 
load_mem_limit
: You can set this property per load job in the 
PROPERTIES
clause. For example:
Copy code
sql
    PROPERTIES ("load_mem_limit" = "4294967296") -- Limit this job to 4GB
This prevents a single massive load from consuming all available memory on a CN and crashing it. 4. Configuration options to limit memory consumption The following CN/BE configurations (in 
cn.conf
) are critical for bounding memory: * 
load_process_max_memory_limit_percent
: Defaults to 30%. This is the total memory all load jobs combined can use on a single node. If your CNs have 64GB RAM, only ~19GB will be used for loading; the rest is reserved for queries. * 
load_process_max_memory_limit_bytes
: A hard byte limit for all load jobs. StarRocks uses the smaller of the percent or byte limit. * 
write_buffer_size
: The size of the memory buffer before data is flushed to disk (default 100MB). If you have high parallelism, many buffers are created. Reducing this can save memory but may create more small files. 5. Can Broker Load memory be bounded more effectively? The most effective way to bound memory and prevent OOMs is a combination of: 1. Lowering the Per-Job Limit: Always specify 
load_mem_limit
in your 
PROPERTIES
for very large loads. 2. Resource Groups (Workload Management): In StarRocks 3.x+, you can use Resource Groups to isolate "Load" workloads from "Query" workloads. You can cap the memory and CPU a specific resource group can use, ensuring a heavy load job is killed or throttled before it takes down the node. 3. Monitor via 
information_schema.loads
: Check the 
MEM_USAGE
of active jobs to identify which specific labels are memory-intensive. Summary Recommendation: If you are hitting OOM, first check if your files are GZIP. If they are, try uncompressed or Parquet. Then, set a strict 
load_mem_limit
in your Broker Load statement and ensure 
load_process_max_memory_limit_percent
is set to a safe value (e.g., 30-50%). References • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md
m
@Rocky in this doc says by default 30% - docs.starrocks.io/docs/…/Memory_management#… The maximum percentage of memory resources that can be taken up by all load processes on a BE node. Its value is the smaller one between 
mem_limit * load_process_max_memory_limit_percent / 100
and 
load_process_max_memory_limit_bytes
. If this threshold is exceeded, a flush and backpressure will be triggered. then what value we need to change then ?
r
To address OOM issues during Broker Loads, you need to determine if your nodes are crashing (the Linux OOM killer is stopping the process) or if the load jobs are failing with a "Memory limit exceeded" error. Depending on your situation, you should adjust the following values: 1. To Prevent Node Crashes (OOM Killer) If your CN/BE nodes are crashing, it means the system is over-committing memory. You should restrict how much memory loading can take to leave room for the OS and other processes. * Decrease 
load_process_max_memory_limit_percent
: Lower this from the default 30 to 20 or 15. This forces StarRocks to trigger "backpressure" (slowing down the read) and "flush" (writing data to disk) earlier, preventing the process from growing too large. * Check 
mem_limit
: Ensure your 
mem_limit
(default 90%) is not too high for your machine. If you have other services on the node, set this to a lower percentage (e.g., 
80%
) or a fixed value (e.g., 
100G
). 2. To Prevent Load Job Failures ("Memory limit exceeded") If your node is healthy but your load job fails with a memory error, the job is hitting a "soft" or "hard" limit. * Increase 
load_process_max_memory_limit_percent
: If your node has plenty of free RAM and you want to allow more of it for loading, increase this to 40 or 50. * Adjust 
load_mem_limit
per job: If a single large job is failing, do not change the global BE config. Instead, increase the limit for that specific job in the SQL statement:
Copy code
sql
    LOAD LABEL my_label ...
    PROPERTIES (
        "load_mem_limit" = "8589934592" -- Increase to 8GB for this job
    );
3. Which one "wins"? StarRocks calculates the total memory allowed for all loading jobs on a node as:
Limit = 
min(
load_process_max_memory_limit_bytes
, 
mem_limit
 * 
load_process_max_memory_limit_percent
 
)
By default, 
load_process_max_memory_limit_bytes
is very large (100GB), so the 
load_process_max_memory_limit_percent
 (30%) is usually the effective bottleneck. Summary: What to change? Goal | Parameter to Change | Action Stop node crashes | 
load_process_max_memory_limit_percent
| Decrease (e.g., to 20) Stop node crashes | 
mem_limit
| Decrease (e.g., to 80%) Allow larger loads | 
load_process_max_memory_limit_percent
| Increase (e.g., to 50) Restrict a specific job | 
load_mem_limit
(in PROPERTIES) | Set (e.g., "4G") Note: If you are using StarRocks 3.3.2 or later, you can also set 
enable_new_load_on_memory_limit_exceeded = false
. This will reject new load jobs if the memory limit is already reached, protecting the nodes from being overwhelmed by too many concurrent jobs. References • administration/management/BE_configuration.md • administration/management/resource_management/Memory_management.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md References • administration/management/BE_configuration.md • administration/management/resource_management/Memory_management.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md
m
@Rocky we get this error too - SQL Error [1064] [42000]: Memory of process exceed limit. read and decompress page Backend: starrocks-cn-2.starrocks-cn-search.abc-db.svc.cluster.local, Used: 20886219368, Limit: 20873541058. Mem usage has exceed the limit of BE: BE:11149
r
This error confirms that your entire Compute Node (CN) process has hit its total memory limit. Based on the numbers provided (
Used: 19.45 GB, Limit: 19.44 GB
), your CN node likely has about 21.6 GB of physical RAM (since StarRocks defaults to a 
mem_limit
of 90%). The error "read and decompress page" typically happens when the CN is scanning data from S3 or writing it to the local sink and doesn't have enough remaining "overhead" memory to perform the decompression/buffer operations. Here is how to fix this specifically for your Broker Load: 1. Set a Hard Limit for the Specific Load Job The most direct way to stop a single large load from taking down the whole node is to cap its memory in the load statement itself. Add the 
load_mem_limit
property to your Broker Load SQL:
Copy code
sql
LOAD LABEL my_heavy_load_job
...
PROPERTIES (
    "load_mem_limit" = "8589934592" -- Limit this job to 8GB (8*1024*1024*1024)
);
By setting this, StarRocks will trigger backpressure (slow down reading) or fail the job specifically, instead of allowing it to consume all 19.44 GB and hitting the process limit. 2. Protect the Node (Change the 30% limit) You previously asked what value to change in the docs. To prevent the load from pushing the node to the "Process limit," you should decrease the 
load_process_max_memory_limit_percent
. If you set this to 20%, the node will only allow all load jobs combined to use ~4.3 GB (20% of your ~21.6 GB RAM). This leaves plenty of headroom for the OS and "read and decompress" overhead. * Action: In your 
cn.conf
(on the CN node) or via SQL if supported:
Copy code
sql
    -- Note: This is usually a CN/BE config file change, not a global SQL variable
    -- Update cn.conf:
    load_process_max_memory_limit_percent = 20
3. Check for File Compression Spikes If your data on S3 is compressed with GZIP, StarRocks cannot split those files. A single CN node will attempt to decompress the entire file into memory. * The Problem: A 2GB GZIP file might expand to 10GB-15GB in memory during processing. * The Fix: Use Parquet or ORC formats, or keep CSV files uncompressed but smaller (e.g., 200MB each). 4. Reduce Parallelism If multiple threads are trying to load data at once, each thread consumes its own buffer memory. * Parameter: 
load_parallel_instance_num
(Default is often 0, which means the system decides). * The Fix: In your Broker Load properties, try setting:
Copy code
sql
    PROPERTIES (
        "load_parallel_instance_num" = "1"
    );
This reduces speed but significantly lowers the memory footprint. Summary of the Situation Your CN is running "too lean" (only ~20GB RAM). When a large load job starts: 1. The load job uses memory for data buffers. 2. The system needs additional memory for "read and decompress" tasks. 3. Because the load job isn't capped (it thinks it can use up to 19GB), it takes everything, leaving nothing for the decompression task. Recommendation: Start by adding 
"load_mem_limit" = "4294967296"
(4GB) to your 
PROPERTIES
and see if the job completes. If it does, you know 4GB is a safe boundary for your current CN size. References • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • administration/management/BE_configuration.md • administration/management/resource_management/Memory_management.md • faq/Sql_faq.md • faq/loading/Broker_load_faq.md References • faq/loading/Broker_load_faq.md
This error is very specific: it means the entire StarRocks process (the CN node) has hit its safety ceiling (
mem_limit
). Even though you have configured a 30% limit for loading, the node is crashing because the total memory (Loading + Metadata + Query + System overhead) has reached 100% of what StarRocks is allowed to use. Here is the breakdown of what to change to fix this: 1. The Value to Change: 
load_process_max_memory_limit_percent
 You should decrease this value if your nodes are crashing. * Current Default: 30% * Recommended Action: Change it to 20 or 15 in your 
cn.conf
. * Why? The error "read and decompress page" happens when StarRocks tries to allocate a small buffer to handle data, but it finds that the memory is already "full." By lowering the load limit to 20%, you force the load process to "backpressure" (slow down) much earlier, leaving 10% more "breathing room" for these decompression tasks and other system operations. 2. The Per-Job "Circuit Breaker": 
load_mem_limit
 The 30% global limit is shared by all load jobs. If you have one "huge" job, it might be trying to take all of that 30%. You should cap it specifically in your SQL command:
Copy code
sql
LOAD LABEL my_huge_data_load
...
PROPERTIES (
    "load_mem_limit" = "4294967296" -- Cap this specific job at 4GB
);
This ensures that if the job starts getting too large, only that job fails, rather than the entire CN node hitting the process limit and failing. 3. Check for "Primary Key" Table Memory (If applicable) If you are loading into a Primary Key table, the memory usage is significantly higher because the CN must keep a primary key index in memory to handle upserts. * If you are hitting OOM on Primary Key loads, lowering the 
load_parallel_instance_num
to 
1
or 
2
is often necessary to prevent too many parallel index lookups. 4. Adjust the Process Limit for Kubernetes (
mem_limit
) In your error, the Limit is ~19.44 GB. * If your Kubernetes Pod has 20 GB of RAM, StarRocks is trying to use 19.44 GB (90%). This is too tight for K8s. The K8s sidecar and the OS need more than 0.5 GB. * Action: Decrease 
mem_limit
in 
cn.conf
to 
80%
or set it to a fixed value like 
18G
. This makes StarRocks more "conservative" and triggers its internal memory management before the process actually hits the hard physical limit. Summary of what to change: 1. 
cn.conf
: Set 
load_process_max_memory_limit_percent = 20
(to trigger flushes earlier). 2. 
cn.conf
: Set 
mem_limit = 80%
(to leave more room for the OS/K8s). 3. SQL Properties: Add 
"load_mem_limit" = "4G"
to your 
BROKER LOAD
statement. 4. SQL Properties: Add 
"load_parallel_instance_num" = "1"
if you continue to hit memory limits, as this reduces the number of concurrent buffers. References • administration/management/BE_configuration.md • administration/management/resource_management/Memory_management.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • faq/loading/Broker_load_faq.md • loading/StreamLoad.md References • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • administration/Meta_recovery.md • faq/loading/Broker_load_faq.md • loading/StreamLoad.md
m
@Rocky I have recently upgraded to 4.1.1 and noticed that "DataCacheMetrics" DiskUsage: 0B/0B always so is this expected? This used to show in 3.5 which is our previous version so what has changed and how to get this working meaning showing so help. Status: Normal, DiskUsage: 0B/0B, MemUsage: 102.2KB/2GB Status: Normal, DiskUsage: 0B/0B, MemUsage: 103.3KB/2GB Status: Normal, DiskUsage: 0B/0B, MemUsage: 109.5KB/2GB
r
In StarRocks v4.0 and later (including your version 4.1.1), the Data Cache system underwent a major refactor to unify "Block Cache" (used for data lakes) and "Page Cache" (used for local storage) into a single management framework. The reason you see 
DiskUsage: 0B/0B
is that in this new unified system, the disk quota now defaults to 
0
 if not explicitly configured. In version 3.5, StarRocks often automatically allocated disk space (defaulting to 80% of the disk), but v4.x requires explicit limits to prevent multiple processes from competing for the same disk. To get the disk cache working again, follow these steps: 1. Set the Disk Quota You must explicitly define how much disk space the Data Cache is allowed to use. Add or update the following parameter in your 
be.conf
(for BE nodes) or 
cn.conf
(for Compute Nodes):
Copy code
properties
# Example: Allocate 100GB for disk cache
datacache_disk_size = 100G
_Note: If you have multiple disks, this value applies per disk or as a total depending on your 
storage_root_path
setup. If you set it to 
0
(the default), disk caching is disabled and only memory is used._ 2. Verify the Storage Path Ensure that 
storage_root_path
is correctly configured. In v4.1, the data cache stores its files under a sub-directory named 
datacache
within the paths defined here:
Copy code
properties
storage_root_path = /path/to/data1;/path/to/data2
3. Check Automatic Adjustment (Alternative) If you want StarRocks to automatically manage the disk space based on available capacity (similar to the old behavior), you can enable the auto-adjust feature:
Copy code
properties
datacache_auto_adjust_enable = true
However, for production environments, explicitly setting 
datacache_disk_size
is recommended to ensure stability and predictable disk usage. 4. Apply Changes 1. Update the 
be.conf
or 
cn.conf
file on all nodes. 2. Restart the BE/CN processes. 3. Verify the status using:
Copy code
sql
   SHOW BACKENDS\G
   -- OR
   SHOW COMPUTE NODES\G
You should now see the second value in 
DiskUsage
reflect your configured limit (e.g., 
0B/100GB
). Summary of Changes (3.5 vs 4.1) Feature | 3.5 (Previous) | 4.1 (Current) Unified Cache | Separate (Block vs Page) | Unified "Data Cache" Default Disk Quota | Often 80% (Automatic) | 0 (Disabled by default) Primary Config | 
starlet_star_cache_disk_size_percent
| 
datacache_disk_size
Troubleshooting If the values still show 
0B/0B
after a restart: • Check the BE/CN logs (
<http://be.INFO|be.INFO>
) for lines containing "DataCache" or "star_cache". It will often report if it failed to initialize the disk cache due to insufficient permissions or invalid paths. • Run 
SELECT * FROM information_schema.be_configs WHERE NAME LIKE '%datacache%';
to verify the node has actually picked up the new configuration values. References • using_starrocks/caching/block_cache.md • data_source/data_cache_troubleshooting.md • data_source/data_cache_observe.md • administration/management/BE_configuration.md • data_source/data_cache.md References • using_starrocks/caching/block_cache.md • sql-reference/System_variable.md • data_source/data_cache_troubleshooting.md • data_source/data_cache.md
m
@Rocky so do you think we need to fix it in order to allocate diskUsage based on _`storage_root_path. or 4.X will show in different metrics or some config values?`_
r
In StarRocks 4.x, the "Data Cache" (formerly Block Cache) underwent a significant refactor to simplify configuration. The 
DiskUsage: 0B/0B
you are seeing indicates that no disk quota has been allocated, which is now a common side effect of the upgrade if you were relying on deprecated parameters or default behaviors from version 3.5. To answer your question: Yes, you likely need to explicitly configure the disk allocation to get it working again. It is not just a display issue; the cache is currently not using any disk space. Why did this change in 4.x? 1. Deprecated Parameters: In v4.0, older parameters like 
block_cache_disk_size
and 
block_cache_enable
were deprecated in favor of 
datacache_enable
and 
datacache_disk_size
. If your 
be.conf
used the old names, 4.x may ignore them, leading to a 
0B
quota. 2. Unified Cache Logic: Version 4.x unified the caching for Data Lake (external) and Shared-Data (internal) tables. This unified system is more conservative about auto-allocating disk space to avoid conflicts with local storage. 3. Default Quota is 0: While 
datacache_auto_adjust_enable
is 
true
by default, if the system detects the disk is already above the "low level" threshold or if specific paths aren't explicitly sized, it may initialize with a 0B quota to protect the node from disk-full errors. How to fix it (and show the metrics) You should manually set the disk quota to ensure the unified cache has a defined boundary. Update your 
be.conf
(or 
cn.conf
if using Compute Nodes) with the following: Option A: Set a fixed size (Recommended) This is the most stable way to ensure the cache works as expected.
Copy code
properties
# Enable the unified data cache
datacache_enable = true
# Allocate a specific amount of disk space (e.g., 100GB)
datacache_disk_size = 100G
Option B: Set a percentage of the disk If you prefer the "percentage of available space" behavior from 3.5:
Copy code
properties
# Example: Use 80% of the disk capacity for cache
starlet_star_cache_disk_size_percent = 80
*Note: In 4.x, the system takes the greater of 
datacache_disk_size
and 
starlet_star_cache_disk_size_percent
.* Where does it store the data? It still uses 
storage_root_path
. In 4.x, the cache files are typically located in a sub-directory: 
${storage_root_path}/datacache/
How to verify without a restart You can try to apply this change dynamically to a single node to see if the metrics update:
Copy code
sql
-- Apply to all backends (not persistent after restart)
UPDATE be_configs SET VALUE="200G" WHERE NAME="datacache_disk_size";
After running this, check 
SHOW BACKENDS\G
again. You should see the second 
0B
change to your configured value (e.g., 
0B/200GB
), and usage should begin to climb as you run queries. Summary of key config changes in 4.x: 3.5 Config (Old) | 4.x Config (New) 
block_cache_enable
| 
datacache_enable
block_cache_disk_size
| 
datacache_disk_size
block_cache_mem_size
| 
datacache_mem_size
(Implicit default) | 
datacache_auto_adjust_enable
(default true) References • release_notes/release-4.0.md • using_starrocks/caching/block_cache.md • data_source/data_cache.md • release_notes/release-3.2.md
Open in Slack
PreviousNext
Powered by