Is it possible to preemptively load data into the datacache in CN nodes for

shared-data

configuration? My usecase is, I have a tiered dataset, partitioned on timestamp, I want the first few months to be

hot

and preloaded into the cache as soon as data is loaded and the rest of the dataset can be

cold

where the penalty of reading from S3 is fine. I dont want to use

shared-nothing

configuration because the dataset is quite large and will become expensive.

check if

cache select

can help here.

@Kevin Cai when the cache select is run, does it put the same data into all nodes, or does the data get distributed evenly between CN nodes?.

it acts as if the query were running, and the data will be cached the same way as cold query filling in the cache.

@Satya Kuppam You can also use `datacache.partition_duration`: The validity duration of the hot data. When the local disk cache is enabled, all data is loaded into the cache. When the cache is full, StarRocks deletes the less recently used data from the cache. When a query needs to scan the deleted data, StarRocks checks if the data is within the duration of validity. If the data is within the duration, StarRocks loads the data into the cache again. If the data is not within the duration, StarRocks does not load it into the cache. This property is a string value that can be specified with the following units:

YEAR

,

MONTH

,

DAY

, and

HOUR

, for example,

7 DAY

and

12 HOUR

. If it is not specified, all data is cached as the hot data. note This property is available only when

datacache.enable

is set to

true

yes

Got it thanks, let me try that and get back to you. Thanks.