@Rocky I'm using 4.0.8 shared data cluster, I have a table with datacache.enable=true, I'm trying to alter table by setting property to false. It is not allowing. Is there a way to not cached the data of the table going forward?

In StarRocks (shared-data mode), the

datacache.enable

table property is typically immutable once the table is created. This is why your

ALTER TABLE

statement is failing. To stop caching data for this table going forward without recreating it, you can use the following workarounds: 1. Modify

datacache.partition_duration

While

datacache.enable

is immutable, the

datacache.partition_duration

(formerly

storage_cache_ttl

) property is mutable in shared-data clusters (v3.1+). This property defines the "hot data" window. Any data older than this duration will not be loaded into the cache during queries. You can set this to a minimal value (the smallest documented unit is

HOUR

, though some versions may support

0

or smaller units):

sql
ALTER TABLE your_table_name SET ("datacache.partition_duration" = "1 HOUR");

By setting a very short duration, most of your table's data will be treated as "cold" and will not be cached when accessed. 2. Use Session Variables If you want to prevent queries from using or populating the cache for this table, you can set session variables before running your queries or within your application session: * To disable reading from the cache:

sql
    SET enable_scan_datacache = false;

* To prevent the query from populating (writing to) the cache:

sql
    SET populate_datacache_mode = 'never';

_Note: In StarRocks 3.3+,

populate_datacache_mode

can be set to

never

,

auto

, or

always

._ 3. Clear Existing Cache (Manual) If you want to free up the disk space currently occupied by this table's cache on the CN/BE nodes, you cannot target a specific table easily via SQL. However, you can manually clear the

datacache

directory on the nodes (though this clears the cache for all tables on that node): 1. Locate the cache directory (defined by

storage_root_path

in

cn.conf

or

be.conf

). 2. The data is usually stored in a sub-directory named

datacache/

. 3. Deleting this directory and restarting the node will clear the cache. Warning: This will cause a temporary performance drop for all tables as the cache is rebuilt. Summary If you must strictly disable caching for this table at the metadata level, the only way is to recreate the table (e.g.,

CREATE TABLE ... AS SELECT

) with

"datacache.enable" = "false"

. Otherwise, setting the

partition_duration

to a minimum is the most effective live workaround. References • deployment/shared_data/gcs.md • deployment/shared_data/s3.md • deployment/shared_data/feature-support-shared-data.md • deployment/shared_data/minio.md

@Rocky can I just enable cache for specific partitions?

In StarRocks shared-data clusters, you cannot explicitly toggle the

datacache.enable

property for specific partitions.

datacache.enable

is a table-level property that is immutable after table creation. However, you can achieve the same result using the following methods: 1. Use

datacache.partition_duration

(Recommended) This property defines the "hot data" window. If your table is partitioned by a time-related column (DATE or DATETIME), StarRocks uses this property to decide which partitions to cache. * How it works: StarRocks compares the partition's time range to the current time. If the partition falls within the

datacache.partition_duration

window, it is considered "hot" and will be cached. If it is older, it is considered "cold" and will not be cached during queries. * How to apply: Even though it is a table-level property, it effectively enables/disables cache based on the partition's value.

sql
    -- Set cache duration to the last 7 days.
    -- Only partitions within the last 7 days will be cached going forward.
    ALTER TABLE your_table_name SET ("datacache.partition_duration" = "7 DAY");

2. Manually Warm Up Specific Partitions If you want to ensure specific partitions are in the cache regardless of their age, you can use the Data Cache Warmup feature (introduced in v3.3). This is an active way to "enable" cache for a specific partition:

sql
-- This proactively loads the specified partition into the local disk cache
CACHE SELECT * FROM your_table_name PARTITION (p20230801);

3. Query-Level Control If you want to prevent certain queries (e.g., those targeting "old" partitions) from populating or using the cache, you can use session variables. This gives you manual "partition-like" control at the application level:

sql
-- Disable cache usage/population for the current session or query
SET enable_scan_datacache = false;
SELECT * FROM your_table_name WHERE partition_col = 'old_partition';

Summary of Constraints *

datacache.enable

: Must be set at

CREATE TABLE

. It cannot be modified later and cannot be set per partition. *

datacache.partition_duration

: Can be modified via

ALTER TABLE ... SET

. It is the primary tool for automatic partition-based caching. * Existing Cache: Changing these properties only affects future cache behavior. To remove data already in the cache for "cold" partitions, you must wait for it to be evicted naturally (LRU) or manually clear the cache directory on the nodes. References • deployment/shared_data/feature-support-shared-data.md • sql-reference/sql-statements/table_bucket_part_index/CREATE_TABLE.md • deployment/shared_data/s3.md References • sql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md • deployment/shared_data/s3.md • deployment/shared_data/minio.md

@Rocky datacache.partition_duration isn't this the validity of cached data of a table?

You are correct that

datacache.partition_duration

(previously called

storage_cache_ttl

) defines the validity window for "hot" data. However, its behavior is specifically designed to allow "partition-level" control over what stays in the cache. Here is how it works and how it answers your goal of caching only specific partitions: 1. It acts as a "Hot Data Window" Instead of a simple "Time to Live" (TTL) that deletes data after X hours, this property defines a rolling window relative to the current time. * For Partitioned Tables: StarRocks checks the partition value (the date/time range of the partition). If the partition's range falls within the

partition_duration

(e.g.,

LAST 7 DAYS

), it is considered "hot." * For Non-Partitioned Tables: It typically uses the data's creation/modification time. 2. How it controls what is cached * If data is "Hot": When you query it, StarRocks will fetch it from object storage and save it to the local disk cache. If it was already in the cache, it stays there. * If data is "Cold": (Outside the duration window): StarRocks will read it directly from object storage but will NOT populate the local cache with it. If it was already in the cache, it will eventually be evicted to make room for hot data. Can you use it for specific partitions? Yes, but only by time. Since you are using a shared-data cluster, this is the intended way to manage cache at a granular level. * If you set

"datacache.partition_duration" = "1 MONTH"

, StarRocks automatically treats the most recent month of partitions as "cacheable" and everything older as "not cacheable." * This effectively "disables" the cache for all partitions older than one month. What if you want to cache a specific non-recent partition? If you have a specific old partition (e.g.,

p2023_Q1

) that you want to keep cached even though it's outside the

partition_duration

, you can use the Manual Warmup command:

sql
-- This bypasses the partition_duration and forces the data into cache
CACHE SELECT * FROM table_name PARTITION (p2023_Q1);

Summary for your 4.0 (3.x) Cluster: To "enable cache for specific partitions," your best tools are: 1. Set a Global Window: Use

ALTER TABLE ... SET ("datacache.partition_duration" = "X DAY")

to cover your most active data. 2. Manual Override: Use

CACHE SELECT

for specific historical partitions you need to speed up temporarily. Note: Since there is no StarRocks 4.0 stable release yet (current is 3.3/3.4), if you are on a very new dev build, these properties are the standard way shared-data (Cloud Native) tables handle caching. References • deployment/shared_data/s3.md • deployment/shared_data/feature-support-shared-data.md • sql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md References • sql-reference/System_variable.md • deployment/shared_data/feature-support-shared-data.md • administration/management/BE_configuration.md • sql-reference/sql-statements/table_bucket_part_index/SHOW_PARTITIONS.md • administration/management/FE_configuration.md • deployment/shared_data/s3.md