@Rocky I have shared data architecture starrocks 4.0.8 running with cache enabled on tables. Lets say If some tablet blocks are cached on the disk, if I query after 1hour, will it get from cache or look at bucket for latest data? how is it controlled the freshness of data when cache is enabled?

In StarRocks shared-data architecture (v4.0.x), the cache does not compromise data freshness. The system is designed to always serve the latest data regardless of whether the cache is enabled. Here is how the freshness and cache logic work for your scenario: 1. The "1 Hour Later" Scenario If you query a table after 1 hour, StarRocks will always return the latest data. Whether it comes from the local disk cache or the bucket depends on whether the data has changed during that hour: * If the data has NOT changed: The Frontend (FE) metadata will point to the same segment files. Since these blocks are already cached on your BE/CN disks, StarRocks will read them directly from the cache to provide high performance. * If the data HAS changed (New Load/Update): StarRocks uses a versioned metadata system. When new data is loaded or a compaction occurs, the FE generates a new version of the metadata pointing to new, immutable segment files in the bucket. Because these new files have different names or internal IDs, the BE will realize they are not in the local cache and will fetch the latest data from the bucket (and subsequently cache it). 2. How Freshness is Controlled Data freshness in StarRocks is controlled by the FE Metadata, not by a "time-to-live" check on the cached files themselves. * Immutability: Segment files in the storage bucket are immutable. Once written, they never change. This eliminates the risk of a "stale" cache file being different from the version in the bucket. * Atomic Metadata Updates: When a transaction (like an

INSERT

) commits, the FE atomically updates the table's version. Every query first consults the FE to get the current list of active segment files. * Consistency: Because the BE only queries the specific file versions requested by the FE, it is impossible for the cache to return "old" data for a table that has been updated. 3. Cache Retention Strategy You mentioned "how is it controlled," and there is one specific property that affects which data stays in the cache: *

datacache.partition_duration

: This table property defines the "hot data" window. For example, if set to

7d

, StarRocks will prioritize caching blocks from partitions created within the last 7 days. Data older than this may not be cached or will be evicted sooner. * LRU Eviction: Aside from the partition duration, the cache is managed by an LRU (Least Recently Used) mechanism. If the disk cache reaches its limit (controlled by

datacache_disk_size

), older or less frequently accessed blocks are evicted to make room for new ones. Summary * Will it get from cache? Yes, if the data is still the "latest" version according to the FE. * Will it look at the bucket? Yes, if new data has been committed or if the cached blocks were evicted due to space constraints. * Freshness guarantee: You always get the latest data. The cache only accelerates the retrieval of the files that the FE identifies as current. Note: In StarRocks 4.0, the "Data Cache" system uniformly manages both Page Cache (memory) and Block Cache (disk) for even better efficiency. References • deployment/shared_data/feature-support-shared-data.md • administration/management/BE_configuration.md • data_source/data_cache.md • data_source/data_cache_troubleshooting.md