Hi! I hope this is the right place for this question! I think I have a bad misunderstanding of how caching works in StarRocks. I am on version 4.0.7 and probably have been mixing up documentation. I am in a shared nothing mode architecture with local NVME disks. Looking at the backend nodes I have configured I see there is a DataCache for memory and disk. You can also configure that data cache disk and data cache mem. However I keep seeing references to page cache and block caches as well. It looks like those concepts were unified under DataCache or am I misunderstanding something? It also says in the DataCache document thats its for remote storage only "Data Cache loads data from remote storage in blocks"... I just want to make sure I take advantage of caching since I have some pretty heavy repeated scans and I would like to ontop of the OS Cache leverage whatever StarRocks offers. I just would like some clarification on what levers to tune correctly...

Hello! I'm Rocky, your StarRocks assistant. I'd be happy to help you! Please let me know what you're working on or what specific issue you're encountering. Whether it's a SQL syntax error, a configuration question, or a performance tuning issue, feel free to share the details!

Since I did this wrong here is the question @Rocky "Hi! I hope this is the right place for this question! I think I have a bad misunderstanding of how caching works in StarRocks. I am on version 4.0.7 and probably have been mixing up documentation. I am in a shared nothing mode architecture with local NVME disks. Looking at the backend nodes I have configured I see there is a DataCache for memory and disk. You can also configure that data cache disk and data cache mem. However I keep seeing references to page cache and block caches as well. It looks like those concepts were unified under DataCache or am I misunderstanding something? It also says in the DataCache document thats its for remote storage only "Data Cache loads data from remote storage in blocks"... I just want to make sure I take advantage of caching since I have some pretty heavy repeated scans and I would like to ontop of the OS Cache leverage whatever StarRocks offers. I just would like some clarification on what levers to tune correctly... "

Hi! I'm Rocky. It sounds like you've been navigating the transition into StarRocks' newer unified caching architecture. You're right that there has been some consolidation of terminology, especially in the v3.3 to v4.0+ range. Here is the clarification on how these caches work in your Shared-Nothing (local NVMe) setup: 1. The Unified "Data Cache" (v4.0+) In the version you are using, StarRocks has unified several concepts under the Data Cache umbrella. It essentially consists of two tiers: * Page Cache (In-Memory): This caches decompressed data pages and index pages in the BE's RAM. * Block Cache (Disk-based): This caches raw data blocks from remote storage (like S3/HDFS) onto local BE disks. 2. Does this apply to Shared-Nothing? This is where the confusion usually lies. * Block Cache (Disk Tier): You are correct—this is primarily for remote storage (Data Lake or Shared-Data mode). Since your data is already on local NVMe disks in a Shared-Nothing architecture, StarRocks will not cache that local data into the Block Cache. Doing so would be redundant (caching local disk to local disk). * Page Cache (Memory Tier): This is heavily used for local tables in Shared-Nothing mode. It stores decompressed pages in memory so that repeated scans don't have to hit the disk or consume CPU cycles for decompression. 3. The "Levers" for Shared-Nothing Performance Since you have heavy repeated scans on local NVMe, you should focus on these three layers: A. StarRocks Page Cache (The Memory Lever) This is the most important lever for you. It sits above the OS and stores the actual data pages your queries need. *

datacache_mem_size

: This is the unified parameter in v4.0. It sets the maximum memory for the Page Cache. You can set this as a percentage (e.g.,

20%

) or a fixed size (e.g.,

100G

). *

disable_storage_page_cache

: Ensure this is set to

false

(it is by default). B. OS Page Cache Since StarRocks uses standard I/O for reading local data files, the Linux OS will automatically cache "hot" file blocks in the remaining free RAM of your server. * Tip: Don't allocate 100% of your machine's RAM to the StarRocks BE process (

mem_limit

). Leave some room (e.g., 10-20%) for the OS Page Cache to handle the raw file I/O. C. Query Result Cache If your "heavy repeated scans" are identical queries, you can bypass the scan entirely by caching the final result set. *

enable_query_cache

: Set this session variable to

true

. This caches the final output of the query in memory. Summary of what to tune: For your specific setup (v4.0.7, Shared-Nothing, NVMe), ignore the

datacache_disk_size

(Block Cache) as it won't help with local tables. Instead, focus on: 1. Increase

datacache_mem_size

: Give more RAM to the Page Cache to keep more decompressed hot data in memory. 2. Monitor Page Cache Hit Rate: Check your BE metrics (via `curl http://be_host:be_http_port/metrics`) for

starrocks_be_page_cache_hit_count

to see if your cache is actually being utilized. 3. Check

storage_page_cache_limit

: In older versions, this was the primary name; in v4.0, it is managed via the unified Data Cache settings, but keeping an eye on both in your

be.conf

is a good idea. To adjust these dynamically for testing:

sql
ADMIN SET FRONTEND CONFIG ("datacache_mem_size" = "20%");
-- Or per BE via the backend configuration API

I hope this clears up the "Remote only" vs "Local" confusion! Let me know if you need help finding specific metrics to verify your hit rates. References • data_source/data_cache.md • administration/management/BE_configuration.md • using_starrocks/caching/block_cache.md