<@U0A71G31CDV> ``` Reason: - MV rewrite ...
# questions-and-troubleshooting
k
@Rocky
Copy code
Reason:
       - MV rewrite fail for my_table_mv: stale partitions refreshType=FULL
why is mv query rewrite failing and why are there stale partitions if the mv refresh is happening faster than the staleness value?
Copy code
DISTRIBUTED BY HASH(`id`) BUCKETS 2048
ORDER BY (id)
REFRESH SCHEDULE EVERY(INTERVAL 2 HOUR)
PROPERTIES (
"replicated_storage" = "true",
"replication_num" = "1",
"mv_rewrite_staleness_second" = "14400",
"datacache.enable" = "true",
"enable_async_write_back" = "false",
"storage_volume" = "shared_data_volume",
"bloom_filter_columns" = "parent_id",
"warehouse" = "default_warehouse"
)
AS SELECT *
FROM iceberg.db.table;
r
The error message
stale partitions refreshType=FULL
indicates that the StarRocks optimizer has disqualified your Materialized View (MV) for query rewrite because it considers the data to be out-of-sync with the base Iceberg table. Even though your MV is not partitioned (unpartitioned), StarRocks internally treats it as having a single logical partition. Since your refresh interval (2 hours) is shorter than your staleness tolerance (4 hours), the rewrite should technically work if the last refresh was successful. Here are the likely reasons why it is failing and how to fix it: 1. Check the Last Refresh Status The most common cause is that the MV has not successfully completed a refresh within the last 4 hours. * Action: Run
SHOW MATERIALIZED VIEWS LIKE 'my_table_mv' \G
and check: * `last_refresh_state`: Must be
SUCCESS
. If it is
FAILED
or
PENDING
, the data is considered stale. * `last_refresh_finished_time`: Compare this to the current time. If this time is more than 4 hours ago (or
NULL
), the
mv_rewrite_staleness_second
threshold is exceeded. * `last_refresh_error_message`: If it failed, this will tell you why (e.g., connectivity issues to Iceberg). 2. Missing External Rewrite Property For MVs built on external catalogs (like Iceberg), StarRocks sometimes requires an explicit property to enable the optimizer to trust the MV for rewrite, especially when dealing with consistency. * Solution: Add
"force_external_table_query_rewrite" = "true"
to your properties.
Copy code
sql
    ALTER MATERIALIZED VIEW my_table_mv SET ("force_external_table_query_rewrite" = "true");
3. Query Rewrite Consistency Level Starting in v3.2, StarRocks introduced
query_rewrite_consistency
. The default is
checked
, which performs strict timing and version checks. If the staleness property isn't being respected as expected, you can try setting this to
loose
. * Solution:
Copy code
sql
    ALTER MATERIALIZED VIEW my_table_mv SET ("query_rewrite_consistency" = "loose");
Note:
loose
will allow the rewrite as long as the MV is
active
, ignoring the staleness clock entirely.
4. Iceberg Specific: Partition Change Detection Even if your MV is unpartitioned, StarRocks tracks the partitions of the base Iceberg table. If the Iceberg table is partitioned and StarRocks detects a change in any base partition, it marks the MV as stale. * If you are on a version earlier than v3.1.4, StarRocks' detection of Iceberg partition changes for rewrite was less mature. * Bug Note: There is a known issue in some versions where an Iceberg "rollback" or specific metadata update can result in a negative staleness calculation, causing the check to fail. Ensure you are on a stable release (v3.1.7+, v3.2.3+, or v3.3+). 5. Recommendation for External Tables For MVs on external data lake tables, it is highly recommended to: 1. Use a Partitioned MV: If the base Iceberg table is partitioned by a time column (e.g.,
ts
), define your MV with
PARTITION BY ts
. This allows StarRocks to only mark specific partitions as stale, potentially allowing query rewrite for the "fresh" historical data even if the latest partition is stale. 2. Verify the
is_active
status
: If the base table schema changed, the MV might be
inactive
. Run
ALTER MATERIALIZED VIEW my_table_mv ACTIVE;
to ensure it is usable. To debug further, run:
Copy code
sql
TRACE REASON MV <your_query>;
This will provide a more detailed explanation of why the optimizer skipped the MV. Referencesusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.mdusing_starrocks/async_mv/use_cases/query_rewrite_with_materialized_views.md
k
@Rocky would starrocks inverted index on shared database materialized view perform well on low cardinality columns (~200 unique values) across 3 billion rows
r
For a column with ~200 unique values across 3 billion rows, an inverted index is generally not recommended and likely won't provide a performance boost over StarRocks' default optimizations. Here is a breakdown of why this is the case, especially in a shared-data (shared-storage) environment: 1. The "Page Density" Problem In StarRocks, the smallest unit of I/O is a Page (usually 64KB). With 3 billion rows and only 200 unique values, each value appears approximately 15 million times. * High Probability of Overlap: Statistically, every single data page in your 3 billion rows will likely contain almost all 200 unique values. * Ineffective Skipping: An index's job is to help the engine skip data pages that don't match your filter. Because your values are so low-cardinality, the index will point to every single page anyway. You will end up loading the index and all the data pages, which is slower than just scanning the data. 2. Built-in Dictionary Encoding StarRocks automatically uses Global Dictionary Encoding for low-cardinality string columns. * It converts strings into small integers and performs "Operation on Encoded Data." * For a column with only 200 values, StarRocks can scan and filter the dictionary-encoded data at near-memory speeds using SIMD instructions. This is usually much faster and more storage-efficient than maintaining an inverted index. 3. Shared-Data Compatibility (Version Matters) If you are using StarRocks in shared-data mode (Standard for the "shared database" mentioned): * v3.3 and earlier: The CLucene-based inverted index is not supported in shared-data clusters. * v4.1 and later: StarRocks introduced a built-in inverted index for shared-data clusters. However, this built-in index is architecturally based on Bitmap Index technology. * Bitmap vs. Inverted: In StarRocks, bitmap/inverted indexes only show significant benefits for low-cardinality columns when you are filtering on multiple columns simultaneously (e.g.,
WHERE color='red' AND size='XL' AND brand='nike'
). For a single-column filter with 200 values, the overhead of reading the index from object storage (S3/GCS/Azure) usually outweighs the gain. 4. When it would perform well You should only consider an inverted index for this column if: 1. Full-Text Search: You need to perform keyword searches (
MATCH
) rather than simple equality (
=
) or
IN
filters. 2. Highly Skewed Data: One value appears 2.99 billion times, and the other 199 values appear only a few thousand times. In this case, the index helps find those rare "needle in a haystack" rows. 3. Complex Multi-column Filters: You frequently filter across 5+ different low-cardinality columns where the intersection of those filters is very small. Recommendation For 3 billion rows and ~200 values: * Skip the Inverted Index: It will likely increase your storage costs and slow down your Materialized View refresh (as the index must be built during the refresh). * Use Sort Keys: If this column is your most frequent filter, put it first in the
ORDER BY
clause of the MV. This will cluster identical values together, allowing the ZoneMap index (automatic) to skip entire data blocks effectively. * Rely on Dictionary Encoding: Let StarRocks’ native vectorized engine handle the low-cardinality scan; it is specifically optimized for this exact profile you described. Referencestable_design/indexes/inverted_index.mdtable_design/indexes/Bitmap_index.mdbest_practices/query_tuning/schema_tuning.md Referencestable_design/indexes/Bitmap_index.md
k
@Rocky would sorting a table by id, where id is a string 11 char alphanumeric, be performant for id IN lookups across a 4 billion table? or should this be supplemented by a bloom for bitmap index
r
For a 4-billion-row table, sorting by an 11-character alphanumeric ID is an excellent choice and is often sufficient on its own. Here is how the performance breakdown works and when you should (or shouldn't) supplement it: 1. Sorting by ID (The Prefix Index) When you define
ORDER BY (id)
, StarRocks automatically creates a Prefix Index (also known as a Sparse Index). * Efficiency: It stores the ID value of every 1,024th row in memory. For 4 billion rows, this index is only ~4 million entries, which fits easily in RAM. * Mechanism: For an
IN
lookup, StarRocks performs a binary search on this in-memory index to point directly to the 64KB data page containing the row. * Suitability: Your 11-char ID is well within the 36-byte limit for the Prefix Index. As long as
id
is the first column in your
ORDER BY
clause, point lookups and
IN
lookups will be extremely fast (typically sub-10ms). 2. Should you add a Bloom Filter? Usually, no. If
id
is the first column in your sort key, a Bloom Filter is redundant for most scenarios. * The "Redundancy" Rule: StarRocks documentation specifically recommends Bloom Filters for high-cardinality columns that are not part of the sort key. * The "Large IN List" Exception: If your queries use very large
IN
lists (e.g., thousands of IDs) where many of the IDs do not actually exist in your table, a Bloom Filter can help. It allows the engine to skip the sparse index search and data page loading entirely for those missing values. * Recommendation: Start without it. If your profile shows high
IOTime
during
IN
lookups with many non-existent keys, then add it:
Copy code
sql
    ALTER TABLE my_table SET ("bloom_filter_columns" = "id");
3. Should you add a Bitmap Index? No. For high-cardinality strings like unique IDs: * Storage Overhead: A Bitmap index on 4 billion unique values will be massive and significantly slow down your data ingestion/refresh. * Performance: Bitmap indexes are designed for columns with lower cardinality or for complex multi-column filtering. For a single high-cardinality ID lookup, they are less efficient than the native Prefix Index. Summary Recommendation * Primary choice: Set
id
as the first column in your
ORDER BY
clause. This gives you the best "bang for your buck" with zero extra storage overhead. * Avoid Bitmap Indexes: They are not suitable for unique/high-cardinality IDs. * Monitor: If you have massive
IN
lists and need to prune "missing" values faster, only then consider adding a Bloom Filter. Referencestable_design/indexes/Prefix_index_sort_key.mdtable_design/indexes/Bloomfilter_index.mdbest_practices/table_clustering.md Referencesknowledge/usage/best-practice.mdtable_design/indexes/Bloomfilter_index.md
k
Untitled
@Rocky i would expect a query like the profile to execute faster since it's doing a video_id IN and the mv is sorted by video_id Query: Summary: - Query ID: 019eb053-c87f-7dae-ab84-2e3a4082a552 - Start Time: 2026-06-10 065901 (Z) - End Time: 2026-06-10 065901 (Z) - Total: 279ms - Query Type: Query - Query State: Finished - StarRocks Version: 4.1.1-14b7e3f - User: root - Default Db: viewiq - Sql Statement: SELECT curated_video.video_id AS curated_video_video_id, curated_video.channel_id AS curated_video_channel_id, curated_video.api_title AS curated_video_api_title, curated_video.api_description AS curated_video_api_description, curated_video.api_tags AS curated_video_api_tags FROM curated_video WHERE curated_video.video_id IN ('9AbaAYWY93Y',...) - Warehouse: default_warehouse - Sql Dialect: StarRocks - Variables: parallel_fragment_exec_instance_num=1,max_parallel_scan_instance_num=-1,pipeline_dop=0,enable_adaptive_sink_dop=true,enable_runtime_adaptive_dop=false,runtime_profile_report_interval=10,resource_group=default_wg - NonDefaultSessionVariables: {"catalog":{"defaultValue":"default_catalog","actualValue":"iceberg"},"new_planner_optimize_timeout":{"defaultValue":3000,"actualValue":10000},"enable_adaptive_sink_dop":{"defaultValue":false,"actualValue":true},"materialized_view_rewrite_mode":{"defaultValue":"DEFAULT","actualValue":"force"},"enable_profile":{"defaultValue":false,"actualValue":true}} - HitMaterializedViews: curated_video_mv - Collect Profile Time: 0 - IsProfileAsync: true Planner: - -- Parser[1] 0 - -- Total[1] 72ms - -- Analyzer[1] 53ms - -- Lock[1] 0 - -- AnalyzeDatabase[1] 0 - -- AnalyzeTemporaryTable[1] 0 - -- AnalyzeTable[1] 53ms - -- Transformer[1] 0 - -- Optimizer[1] 11ms - -- MVPreprocess[1] 0 - -- MVChooseCandidates[1] 0 - -- MVGenerateMvPlan[1] 0 - -- MVPrepareRelatedMVs[1] 0 - -- MVTimelinessUpdateInfo[1] 0 - -- MVProcessWithView[1] 0 - -- MVTextRewrite[1] 0 - -- RuleBaseOptimize[1] 10ms - -- CostBaseOptimize[1] 0 - -- PhysicalRewrite[1] 0 - -- DynamicRewrite[1] 0 - -- PlanValidate[1] 0 - -- TypeChecker[1] 0 - -- ConditionalTypeChecker[1] 0 - -- CTEUniqueChecker[1] 0 - -- InputDependenciesChecker[1] 0 - -- ColumnReuseChecker[1] 0 - -- ExecPlanBuild[1] 6ms - -- Pending[1] 0 - -- Prepare[1] 0 - -- Deploy[1] 37ms - -- DeployLockInternalTime[1] 37ms - -- DeploySerializeConcurrencyTime[2] 3ms - -- DeployStageByStageTime[6] 1ms - -- DeployWaitTime[6] 31ms - -- DeployAsyncSendTime[13] 0 - -- DeploySerializeTime[1] 0 - -- DeployScanRanges[1] 0 - MVGlobalCacheStats - MVPlanCacheStats: CacheStats{hitCount=1476147, missCount=18, loadSuccessCount=18, loadFailureCount=0, totalLoadTime=13975607336, evictionCount=0, evictionWeight=0} - curated_video_mv: Rewrite Succeed - DeployDataSize: 1092129 Reason: Execution: - Topology: {"rootId":2,"nodes":[{"id":2,"name":"EXCHANGE","properties":{"sinkIds":[],"displayMem":true},"children":[1]},{"id":1,"name":"PROJECT","properties":{"sinkIds":[2],"displayMem":false},"children":[0]},{"id":0,"name":"OLAP_SCAN","properties":{"displayMem":false},"children":[]}]} - FrontendProfileMergeTime: 1.489ms - QueryAllocatedMemoryUsage: 1.861 GB - QueryCumulativeCpuTime: 4s619ms - QueryCumulativeNetworkTime: 37.878ms - QueryCumulativeOperatorTime: 296.910ms - QueryCumulativeScanTime: 85.609ms - QueryDeallocatedMemoryUsage: 1.791 GB - QueryExecutionWallTime: 194.199ms - QueryPeakMemoryUsagePerNode: 8.585 MB - QueryPeakScheduleTime: 193.287ms - QuerySpillBytes: 0.000 B - QuerySumMemoryUsage: 94.749 MB - ResultDeliverTime: 22.613ms Fragment 0: - BackendAddresses: starrocks-cn-8.starrocks-cn-search.starrocks.svc.cluster.local:9060 - InstanceIds: 019eb053-c87f-7dae-ab84-2e3a4082a553 - EnableEventScheduler: true - BackendNum: 1 - BackendProfileMergeTime: 305.015us - InitialProcessDriverCount: 44 - InitialProcessMem: 59.616 GB - InstanceAllocatedMemoryUsage: 2.950 MB - InstanceDeallocatedMemoryUsage: 2.465 MB - InstanceNum: 1 - InstancePeakMemoryUsage: 654.492 KB - JITCounter: 0 - JITTotalCostTime: 0ns - QueryMemoryLimit: -1.000 B Pipeline (id=0): - IsGroupExecution: false - ActiveTime: 766.787us - __MAX_OF_ActiveTime: 2.002ms - __MIN_OF_ActiveTime: 186.455us - BlockByInputEmpty: 50 - __MAX_OF_BlockByInputEmpty: 9 - __MIN_OF_BlockByInputEmpty: 6 - BlockByOutputFull: 76 - __MAX_OF_BlockByOutputFull: 13 - __MIN_OF_BlockByOutputFull: 7 - BlockByPrecondition: 0 - DegreeOfParallelism: 7 - DriverTotalTime: 193.495ms - __MAX_OF_DriverTotalTime: 193.590ms - __MIN_OF_DriverTotalTime: 193.455ms - PeakDriverQueueSize: 116 - __MAX_OF_PeakDriverQueueSize: 20 - __MIN_OF_PeakDriverQueueSize: 15 - PendingTime: 0ns - InputEmptyTime: 153.641ms - __MAX_OF_InputEmptyTime: 165.934ms - __MIN_OF_InputEmptyTime: 147.844ms - FirstInputEmptyTime: 86.046ms - __MAX_OF_FirstInputEmptyTime: 86.047ms - __MIN_OF_FirstInputEmptyTime: 86.046ms - FollowupInputEmptyTime: 67.594ms - __MAX_OF_FollowupInputEmptyTime: 79.887ms - __MIN_OF_FollowupInputEmptyTime: 61.797ms - OutputFullTime: 22.179ms - __MAX_OF_OutputFullTime: 22.613ms - __MIN_OF_OutputFullTime: 21.577ms - ScheduleCount: 133 - __MAX_OF_ScheduleCount: 23 - __MIN_OF_ScheduleCount: 14 - ScheduleTime: 192.728ms - __MAX_OF_ScheduleTime: 193.287ms - __MIN_OF_ScheduleTime: 191.587ms - TotalDegreeOfParallelism: 7 - YieldByLocalWait: 0 - YieldByPreempt: 0 - YieldByTimeLimit: 0 RESULT_SINK (plan_node_id=-1): CommonMetrics: - IsFinalSink - OperatorTotalTime: 116.873us - __MAX_OF_OperatorTotalTime: 346.734us - __MIN_OF_OperatorTotalTime: 32.480us - OutputChunkBytes: 0.000 B - PullChunkNum: 0 - PullRowNum: 0 - PullTotalTime: 0ns - PushChunkNum: 49 - __MAX_OF_PushChunkNum: 27 - __MIN_OF_PushChunkNum: 3 - PushRowNum: 500 - __MAX_OF_PushRowNum: 206 - __MIN_OF_PushRowNum: 3 - PushTotalTime: 112.366us - __MAX_OF_PushTotalTime: 319.904us - __MIN_OF_PushTotalTime: 31.825us UniqueMetrics: - SinkType: MYSQL_PROTOCAL - AppendChunkTime: 56.364us - __MAX_OF_AppendChunkTime: 157.118us - __MIN_OF_AppendChunkTime: 13.521us - ResultRendTime: 50.587us - __MAX_OF_ResultRendTime: 147.188us - __MIN_OF_ResultRendTime: 10.543us - TupleConvertTime: 44.562us - __MAX_OF_TupleConvertTime: 121.965us - __MIN_OF_TupleConvertTime: 5.713us - NumSentRows: 500 - __MAX_OF_NumSentRows: 206 - __MIN_OF_NumSentRows: 3 CHUNK_ACCUMULATE (plan_node_id=-1): CommonMetrics: - IsSubordinate - OperatorTotalTime: 68.543us - __MAX_OF_OperatorTotalTime: 166.572us - __MIN_OF_OperatorTotalTime: 3.086us - OutputChunkBytes: 162.891 KB - __MAX_OF_OutputChunkBytes: 67.310 KB - __MIN_OF_OutputChunkBytes: 404.000 B - PullChunkNum: 49 - __MAX_OF_PullChunkNum: 27 - __MIN_OF_PullChunkNum: 3 - PullRowNum: 500 - __MAX_OF_PullRowNum: 206 - __MIN_OF_PullRowNum: 3 - PullTotalTime: 1.198us - __MAX_OF_PullTotalTime: 3.857us - __MIN_OF_PullTotalTime: 479ns - PushChunkNum: 445 - __MAX_OF_PushChunkNum: 178 - __MIN_OF_PushChunkNum: 3 - PushRowNum: 500 - __MAX_OF_PushRowNum: 206 - __MIN_OF_PushRowNum: 3 - PushTotalTime: 66.652us - __MAX_OF_PushTotalTime: 162.231us - __MIN_OF_PushTotalTime: 2.045us UniqueMetrics: EXCHANGE_SOURCE (plan_node_id=2): CommonMetrics: - ConjunctsInputRows: 500 - __MAX_OF_ConjunctsInputRows: 206 - __MIN_OF_ConjunctsInputRows: 3 - ConjunctsOutputRows: 500 - __MAX_OF_ConjunctsOutputRows: 206 - __MIN_OF_ConjunctsOutputRows: 3 - ConjunctsTime: 11.027us - __MAX_OF_ConjunctsTime: 27.984us - __MIN_OF_ConjunctsTime: 802ns - JoinRuntimeFilterEvaluate: 0 - JoinRuntimeFilterHashTime: 0ns - JoinRuntimeFilterInputRows: 0 - JoinRuntimeFilterOutputRows: 0 - JoinRuntimeFilterTime: 0ns - OperatorTotalTime: 383.586us - __MAX_OF_OperatorTotalTime: 982.710us - __MIN_OF_OperatorTotalTime: 36.869us - OutputChunkBytes: 162.891 KB - __MAX_OF_OutputChunkBytes: 67.310 KB - __MIN_OF_OutputChunkBytes: 404.000 B - PullChunkNum: 445 - __MAX_OF_PullChunkNum: 178 - __MIN_OF_PullChunkNum: 3 - PullRowNum: 500 - __MAX_OF_PullRowNum: 206 - __MIN_OF_PullRowNum: 3 - PullTotalTime: 379.221us - __MAX_OF_PullTotalTime: 977.079us - __MIN_OF_PullTotalTime: 34.466us - PushChunkNum: 0 - PushRowNum: 0 - PushTotalTime: 0ns - RuntimeFilterNum: 0 - RuntimeInFilterNum: 0 UniqueMetrics: - BufferUnplugCount: 11 - __MAX_OF_BufferUnplugCount: 8 - __MIN_OF_BufferUnplugCount: 0 - BytesPassThrough: 14.844 KB - __MAX_OF_BytesPassThrough: 13.156 KB - __MIN_OF_BytesPassThrough: 0.000 B - BytesReceived: 154.425 KB - __MAX_OF_BytesReceived: 28.429 KB - __MIN_OF_BytesReceived: 17.559 KB - ClosureBlockCount: 0 - ClosureBlockTime: 0ns
r
The 279ms latency you are seeing is actually quite healthy for a query involving Materialized View (MV) rewrite and shared-storage access, but the profile reveals exactly where the time is going. Your query isn't slow because of the scan itself—it's slow because of Frontend (FE) planning overhead and initial I/O wait. 1. The Bottleneck: Frontend Planning (109ms) A significant chunk of your time is spent before the query even starts running on the backends: * Planner (72ms): Because you have
materialized_view_rewrite_mode = "force"
, the FE has to perform a complex logical comparison to ensure the MV is valid and can safely replace the base table. Even for a simple query, this adds 50–70ms of "thinking time" to the FE. * Deploy (37ms): This is the time taken to ship the execution plan to the Compute Nodes (CNs). How to improve this: If you run this specific query frequently, StarRocks' Plan Cache should eventually kick in (you can see
MVPlanCacheStats
in your profile), which would drop that 72ms planning time significantly on subsequent identical queries. 2. Execution Wait: InputEmptyTime (153ms) In the Execution section, your
DriverTotalTime
is 193ms, but
InputEmptyTime
is 153ms. * What this means: The query execution threads were sitting idle for 153ms waiting for the OLAP_SCAN operator to return the first piece of data. * Why it's happening: In a shared-data (Cloud Native) architecture, even with a sort key, the CN must: 1. Fetch the metadata for the data files from the FE or Object Storage. 2. Open the file headers on S3/Blob storage. 3. If the data isn't in the local
datacache
, it has to pull the specific 64KB pages over the network. The Sort Key is working: Your
QueryCumulativeScanTime
is only 85ms. This confirms that the engine successfully used the prefix index to ignore 99.9% of your 4 billion rows and only read the specific blocks containing those IDs. Without the sort key, this scan would have taken minutes, not milliseconds. 3. Large IN List Processing If your
IN
list contains hundreds or thousands of IDs: * The FE has to parse and analyze every single string in that list (contributing to the 53ms
AnalyzeTable
time). * For very long
IN
lists, the engine may convert the
IN
into a Runtime Filter or a hidden Join, which adds a small amount of overhead compared to a single equality filter. Recommendations to hit < 100ms 1. Warm the Cache: Check if your
datacache
is hit. If you run the same query twice and the second one is much faster, the first one was likely waiting on Object Storage I/O. 2. Shorten the IN List: If you are passing thousands of IDs, try to batch them or ensure they are sorted in the query string to match the table's sort order (though the engine usually handles this). 3. Check FE-CN Latency: The
DeployWaitTime
of 31ms suggests a slight delay in network communication between your FE and CN. Ensure they are in the same availability zone/subnet. 4. Use Prepared Statements: If this is a programmatic lookup, using server-side prepared statements can bypass much of the 72ms Planner overhead. Summary: Your sort key is doing its job (85ms scan). The remaining ~200ms is a combination of the "cost of doing business" for MV rewrites (72ms) and the inherent latency of fetching metadata/data in a shared-storage environment.
k
@Rocky how do i make the query operate more in the starrocks mv without going to s3
r
To ensure your queries against the Materialized View (MV) operate as much as possible from local compute node (CN) storage rather than S3, you need to leverage the Data Cache (also known as Block Cache) in StarRocks' shared-data architecture. Here is how you can force the MV data to stay local: 1. Proactive Warmup with
CACHE SELECT
StarRocks v3.3+ introduced an active warmup mechanism. You can proactively pull the MV's data from S3 into the local disks of your compute nodes:
Copy code
sql
-- Warm up the specific MV
CACHE SELECT * FROM curated_video_mv;
If you only query specific columns (like
video_id
and
api_title
), you can warm up only those to save cache space:
Copy code
sql
CACHE SELECT video_id, channel_id FROM curated_video_mv;
2. Automate Warmup after MV Refresh Since an MV's data changes after a refresh, the old cache becomes invalid. You can schedule a task to automatically warm up the cache as soon as the MV is updated:
Copy code
sql
SUBMIT TASK warmup_mv_task 
SCHEDULE EVERY (INTERVAL 1 HOUR) -- Match your MV refresh frequency
AS 
CACHE SELECT * FROM curated_video_mv;
3. Adjust Cache Priority (
partition_duration
)
You can tell StarRocks to prioritize keeping this MV's data in the cache for a longer period. This prevents it from being evicted by other "colder" data:
Copy code
sql
ALTER TABLE curated_video_mv SET ("datacache.partition_duration" = "1 MONTH");
* What this does: It marks the data as "hot" for the specified duration. If the cache is full, StarRocks will avoid evicting this data in favor of data older than one month. 4. Reduce Frontend Rewrite Overhead In your profile, 72ms was spent just on the FE "thinking" about the MV rewrite. You can cache this decision-making process: * Enable Query Context Cache (v3.3+):
Copy code
sql
    SET GLOBAL enable_mv_query_context_cache = true;
This caches the results of the MV rewrite analysis, which can bring that 72ms planning time down to under 10ms for repeated queries. 5. Verify the Cache Status To see if your efforts are working, check the profile of your next query for these specific metrics: *
DataCacheReadBytes
: Should be high (indicates local disk cache hits). *
RemoteReadBytes
: Should be 0 or very low (indicates data was NOT fetched from S3). *
DataCacheReadPeerBytes
: If you have multiple nodes, this shows data fetched from a neighbor's cache instead of S3. Summary Recommendation: 1. Run
CACHE SELECT
on the MV. 2. Set
datacache.partition_duration
to a high value. 3. If this is a high-concurrency app, use
enable_mv_query_context_cache
to eliminate the FE planning bottleneck you FE planning lag. Referencesdata_source/block_cache_warmup.mddeployment/shared_data/feature-support-shared-data.mdadministration/management/FE_configuration.md Referencesdata_source/block_cache_warmup.mddeployment/shared_data/feature-support-shared-data.mdadministration/management/FE_configuration.mddata_source/data_cache.md Referencesdata_source/block_cache_warmup.mdadministration/management/FE_configuration.mdadministration/management/monitoring/metrics-materialized_view.md
k
@Rocky doesnt the entirety of the mv live on disk on starrocks? like: CREATE MATERIALIZED VIEW default_catalog.starrocks.my_mv DISTRIBUTED BY HASH(video_id) BUCKETS 2048 ORDER BY (video_id) REFRESH ASYNC EVERY (INTERVAL 2 HOUR) PROPERTIES ( "bloom_filter_columns" = "video_id,channel_id, "mv_rewrite_staleness_second" = "14400" ) AS SELECT * FROM iceberg.viewiq.my_table;
r
No, in your specific setup (StarRocks 4.x / Cloud Native), the data does not "live" permanently on the local disks of the nodes. Here is the distinction between what is happening and what you likely expect: 1. Internal Storage vs. External Storage When you create an Asynchronous MV, StarRocks indeed creates its own internal physical table. * The Benefit: It is no longer querying the Iceberg table directly. It has its own copy in the StarRocks native format, which is highly optimized for the
ORDER BY
and
video_id
lookups you defined. * The Catch: Because you are using a Shared-Data (Cloud Native) architecture (evident by your
starrocks-cn
nodes), the "permanent" home for this internal table is Object Storage (S3/Azure Blob/etc.), not the local SSDs of the compute nodes. 2. S3 vs. Local Disk (The Cache) In your architecture: * Object Storage (S3): This is where the 4 billion rows of your MV are permanently stored. It is the "Source of Truth." * Local Disk (CN Cache): The disks on your
starrocks-cn-8
node are used only as a temporary cache. When you run your
video_id IN (...)
query, StarRocks looks for those specific data pages on the local disk. If they aren't there (a "cache miss"), the node must go to S3 to fetch them. This is why you see *153ms of `InputEmptyTime`*—the CPU is literally idling while waiting for S3 to send the data. 3. How to make it "Live" on the local disk If you want the MV to behave as if it were entirely on local disk (sub-10ms latency), you have two options: Option A: Active Warmup (Recommended) You can force StarRocks to pull the MV data from S3 into the local compute node disks before you run your queries.
Copy code
sql
-- This moves the data from S3 to the local CN cache
CACHE SELECT * FROM curated_video_mv;
Once this finishes, your
IN
lookups will read from local SSDs instead of S3. Option B: Increase Cache Persistence By default, the cache follows an LRU (Least Recently Used) policy. If other queries run, they might kick your MV data out of the cache. You can tell StarRocks to prioritize this MV:
Copy code
sql
ALTER TABLE curated_video_mv SET ("datacache.partition_duration" = "1 MONTH");
Why your query is "slow" (279ms) Your current profile shows that even though you have an MV: 1. FE Overhead (109ms): The Frontend is spending time rewriting the query from the Iceberg table to your MV. 2. I/O Latency (153ms): The Compute Node is fetching the data from S3 because it isn't in the local cache yet. To fix this: Use
CACHE SELECT
to warm the cache and enable the MV Query Context Cache (
SET GLOBAL enable_mv_query_context_cache = true;
) to eliminate the Frontend planning time. Summary The MV is a physical StarRocks table, but in a Shared-Data cluster, that physical table is stored in S3. The "disk" on your StarRocks nodes is just a fast window into that S3 bucket. Referencesintroduction/Architecture.mdusing_starrocks/async_mv/Materialized_view.mddeployment/shared_data/feature-support-shared-data.mddata_source/data_cache.md Referencesusing_starrocks/async_mv/Materialized_view.mdintroduction/Architecture.mddeployment/shared_data/azure.mddeployment/shared_data/feature-support-shared-data.md
k
@Rocky so this query:
Copy code
sql
ALTER TABLE curated_video_mv SET ("datacache.partition_duration" = "1 MONTH");
will prevent my mv to permanently live on disk for a month? what if want it to always persist on disk as long as the mv is defined and refreshing? once i define an mv i want the data to be permanently live on disk
r
In the StarRocks Shared-Data (Cloud Native) architecture you are using, there is a fundamental difference between "Persistent Storage" and "Compute Cache" that you should keep in mind: 1. S3 is the only "Permanent" home In your cluster, the Source of Truth is always S3. When you define an MV, StarRocks writes the data to S3. The local disks on your CN nodes are technically a Data Cache. * Good news: As long as you have enough disk space on your nodes, your MV can live permanently on those disks. * The Catch: Because it is a cache, StarRocks manages it using an LRU (Least Recently Used) policy. If your nodes run out of disk space because you are querying other huge datasets, StarRocks will evict the oldest data from the disk to make room, even if it belongs to your MV. 2. How to "Pin" it as much as possible To make your MV behave as if it were permanently on disk, use these three settings: A. Set a "Forever" TTL By setting a very long duration, you tell StarRocks that this data is always hot. It will never be considered "cold" and will only be removed if the physical disk literally runs out of space.
Copy code
sql
ALTER TABLE curated_video_mv SET ("datacache.partition_duration" = "100 YEAR");
B. Active Warmup after Refresh Since your MV refreshes every 2 hours, the new data is written to S3. To ensure it is "permanently" moved to the local disks immediately after the refresh, you can run:
Copy code
sql
CACHE SELECT * FROM curated_video_mv;
(You can automate this by adding it to a StarRocks Task that runs after your MV refresh). C. Increase Cache Quota Ensure your nodes have a large enough
datacache_disk_size
. If your MV is 500GB and your node only has 400GB of cache space, it is physically impossible for the MV to "permanently live on disk." * Check your current quota:
SHOW BACKENDS;
(Look at
DataCacheMetrics
). * Increase it in
cn.conf
using
datacache_disk_size
. 3. If you want "True" Permanent Disk (No S3) If your requirement is that the data must live on a local disk and never be subject to cache eviction or S3 round-trips, you would need to use the Shared-Nothing (Classic) architecture of StarRocks instead of the Cloud-Native one. * Shared-Nothing: BE nodes own the disks. Data is stored locally first and replicated across nodes. (No S3 involved). * Shared-Data (Your current setup): CN nodes are "stateless" caches. Data is stored in S3 and cached locally for speed. Summary for your 4.1.1 cluster: You cannot "disable" the cache nature of the local disks in a Cloud-Native cluster. However, by setting
partition_duration
to
100 YEAR
and ensuring your physical disks are large enough to hold the MV, the data will stay on the disk indefinitely. It effectively becomes permanent.** If the query is still slow after doing this, the bottleneck is likely the Frontend Planning time (72ms) we saw in your profile. You can fix that by running:
Copy code
sql
SET GLOBAL enable_mv_query_context_cache = true;
Referencesdeployment/shared_data/feature-support-shared-data.mddeployment/shared_data/s3.mdknowledge/admin_and_operation/shared_data_compaction.md
k
@Rocky this is a materialized view i want to be permantnyl hot, it is simply a mirror of an iceberg table:
Copy code
ALTER MATERIALIZED VIEW default_catalog.starrocks.my_mv SET ("datacache.partition_duration" = "100 YEAR");
Query 1 ERROR at Line 6: : Getting analyzing error. Detail message: Modify failed because unknown properties: {datacache.partition_duration=10 YEAR}, please add
session.
prefix if you want add session variables for mv(eg, "session.insert_timeout"="30000000")..
r
The error occurs because
datacache.partition_duration
is a storage property of the underlying table, rather than a metadata property of the Materialized View (MV). In StarRocks, while you use
ALTER MATERIALIZED VIEW
for refresh logic or TTL, storage-related configurations must be applied using the
ALTER TABLE
syntax. 1. Correct Syntax Use
ALTER TABLE
even though the object is a Materialized View:
Copy code
sql
ALTER TABLE default_catalog.starrocks.my_mv 
SET ("datacache.partition_duration" = "100 YEAR");
2. Verify the Change After running the command, you can verify that the property has been applied to the MV's underlying storage by checking the table properties:
Copy code
sql
SHOW CREATE TABLE default_catalog.starrocks.my_mv;
Look for
"datacache.partition_duration" = "100 YEAR"
in the
PROPERTIES
block of the output. 3. Why this happened * Metadata vs. Storage: StarRocks treats the MV as two parts: the "View" (the query and refresh logic) and the "Table" (the actual physical data in StarRocks format). * Property Scope: Properties like
mv_rewrite_staleness_second
or
partition_ttl
are specific to the MV logic and are changed via
ALTER MATERIALIZED VIEW
. Properties like
datacache.*
,
bloom_filter_columns
, or
replication_num
affect how the data is physically stored/cached and often require
ALTER TABLE
. 4. Important: Don't forget the "Warmup" Setting the duration to
100 YEAR
tells StarRocks: "Do not evict this data from the local cache based on its age." However, if the data is not currently on the disk, it still won't be there until it is accessed or manually loaded. To move the data from S3 to your disks right now, run:
Copy code
sql
CACHE SELECT * FROM default_catalog.starrocks.my_mv;
5. Automated "Permanent" Hotness Since your MV refreshes every 2 hours, the new data written to S3 is technically "new." To ensure your local disks always have the freshest data without waiting for the first slow query to pull it from S3, you should run the
CACHE SELECT
command immediately after every refresh. Referencessql-reference/sql-statements/materialized_view/ALTER_MATERIALIZED_VIEW.mddeployment/shared_data/s3.mddata_source/block_cache_warmup.md Referencesusing_starrocks/async_mv/feature-support-asynchronous-materialized-views.mdusing_starrocks/async_mv/use_cases/create_partitioned_materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md
k
@Rocky I get an error: ALTER TABLE default_catalog.starrocks.curated_video_mv SET ("datacache.partition_duration" = "100 YEAR"); Query 1 ERROR at Line 8: : Getting analyzing error from line 1, column 12 to line 1, column 38. Detail message: The 'curated_video_mv' cannot be alter by 'ALTER TABLE', because it is a materialized view,you can use 'ALTER MATERIALIZED VIEW' to alter it..
r
The error you are seeing is a result of how StarRocks manages Materialized Views (MVs). There are two reasons why your commands are failing: 1. Why
ALTER TABLE
failed
In StarRocks, an Asynchronous MV is backed by a physical table, but the system blocks the
ALTER TABLE
command on it to protect the MV's metadata integrity. You are forced to use
ALTER MATERIALIZED VIEW
for any modifications. 2. Why
ALTER MATERIALIZED VIEW
failed ("Unknown Property")
The property
datacache.partition_duration
is specifically designed for partitioned tables. * Your MV is not partitioned: In your
CREATE
statement, you used
DISTRIBUTED BY HASH(video_id)
, but you did not include a
PARTITION BY
clause. * The Constraint: In StarRocks (including version 3.x and 4.x), the
datacache.partition_duration
property is only valid for tables and MVs that use a partitioning strategy (like
PARTITION BY date_column
). Since your MV is a single, non-partitioned block of data, the system does not recognize a "partition duration" property for it. How to achieve "Permanently Live on Disk" If you want the MV data to stay permanently on the local disks of your compute nodes (CNs) and never be evicted to S3, you have two paths: Option A: Partition the MV (Recommended) If your Iceberg table has a date or time column, you should partition the MV. This not only allows you to use the cache duration property but also makes the MV refresh much more efficient (it will only refresh changed partitions).
Copy code
sql
-- 1. Drop the old MV
DROP MATERIALIZED VIEW curated_video_mv;

-- 2. Re-create with partitioning
CREATE MATERIALIZED VIEW curated_video_mv
PARTITION BY date_trunc('day', event_time) -- Example partition column
DISTRIBUTED BY HASH(video_id) BUCKETS 2048
REFRESH ASYNC EVERY (INTERVAL 2 HOUR)
PROPERTIES (
    "datacache.partition_duration" = "100 YEAR"
)
AS SELECT * FROM iceberg.viewiq.my_table;
Once partitioned, you can dynamically change the duration later using
ALTER MATERIALIZED VIEW curated_video_mv SET ("datacache.partition_duration" = "100 YEAR");
. Option B: Warmup without Partitioning If you cannot partition the MV, you cannot use the
partition_duration
property. In this case, the data will be managed by the standard LRU (Least Recently Used) cache policy. To keep it "hot" on disk: 1. Manual Warmup: Run
CACHE SELECT * FROM curated_video_mv;
after every refresh. This forces the CN nodes to pull the data from S3 into the local disks. 2. Increase Cache Size: Ensure your CN nodes have enough
datacache_disk_size
(configured in
cn.conf
) so that the MV data isn't kicked out by other queries. Summary Because your MV lacks a
PARTITION BY
clause, the property
datacache.partition_duration
is rejected. To pin the data permanently, partition your Materialized View by a time-time-related column. Referencesusing_starrocks/async_mv/use_cases/create_partitioned_materialized_view.mdsql-reference/sql-statements/materialized_view/ALTER_MATERIALIZED_VIEW.mddata_source/data_cache.mddeployment/shared_data/feature-support-shared-data.md Referencesusing_starrocks/async_mv/use_cases/create_partitioned_materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md
k
@Rocky my table is a dimensional table. is there a way to not partition on a datetime field
r
For a dimensional table that is not partitioned, you actually don't need to set the
datacache.partition_duration
property at all. Here is the breakdown of why you are getting errors and how to achieve your goal: 1. Why you are getting the "Unknown Property" Error The property
datacache.partition_duration
is specifically designed for Time-Series/Partitioned data. * How it works: It compares the value of your partition column (e.g.,
2024-01-01
) to the current system time to decide if data is "hot" or "cold." * The Problem: Since your dimension table is not partitioned by time, the system has no "date" to compare against, so the property is invalid and rejected. 2. Dimension Tables are "Always Hot" by Default In StarRocks Shared-Data (Cloud-Native), if you do not specify a
partition_duration
, the system defaults to treating all data as hot. * This means your dimension table is already configured to live on the disk permanently (it will never be marked as "cold"). * It will only be removed from the disk if the physical disk becomes 100% full, at which point the LRU (Least Recently Used) policy will evict the oldest data to make room for new queries. 3. How to ensure it stays "Permanently Live on Disk" Since you want to ensure the data is always available locally and never has to go back to S3, follow these steps: Step A: Use
CACHE SELECT
after Refresh
When your MV refreshes every 2 hours, the new data is written to S3. To ensure it is immediately pulled into the local disks of all your compute nodes (not just the one that ran the refresh), run this command as part of your refresh pipeline:
Copy code
sql
CACHE SELECT * FROM default_catalog.starrocks.curated_video_mv;
This forces the cluster to "warm up" the local disk cache with the entire table. Step B: Enable Cache Sharing If you have multiple compute nodes (CNs), you should enable Cache Sharing. This allows Node A to fetch cached data from Node B over the network instead of going all the way back to S3.
Copy code
sql
SET GLOBAL enable_datacache_sharing = true;
Step C: Check your Disk Quota If your dimension table is 100GB and your CN nodes only have 50GB of cache space, it will keep getting evicted. Check your current capacity:
Copy code
sql
SHOW BACKENDS;
-- Look at the DataCacheMetrics column
If
DiskUsage
is near the limit, you need to increase
datacache_disk_size
in your
cn.conf
. Summary for your Dimension Table 1. Do not partition it (keep it as a simple
DISTRIBUTED BY HASH
). 2. Ignore
partition_duration
(it is already "infinite" by default for non-partitioned tables). 3. Run
CACHE SELECT
after your MV refreshes to "pin" it to the local disks. Referencesdeployment/shared_data/s3.mddata_source/block_cache_warmup.mddeployment/shared_data/feature-support-shared-data.mddata_source/data_cache.md Referencesusing_starrocks/caching/query_cache.mddeployment/shared_data/feature-support-shared-data.mdusing_starrocks/caching/block_cache.mddeployment/shared_data/s3.mddata_source/data_cache.md
k
@Rocky If I simply want to keep a remote iceberg table permanently on disk cache what is the best way to do that
r
For a non-partitioned dimension table or Materialized View (MV), you don't actually need the
datacache.partition_duration
property. Here is why you are seeing those errors and the best way to achieve your goal. 1. Why you are getting the errors * Property Constraint: In StarRocks,
datacache.partition_duration
is a logic-based property that requires a partition column (like a
DATE
) to calculate whether data is "hot" or "cold." Since your dimension MV is not partitioned, the system does not recognize this property for that object, leading to the "Unknown properties" error. * Object Type: StarRocks requires you to use
ALTER MATERIALIZED VIEW
for MVs, not
ALTER TABLE
, even for storage-related properties. 2. The Good News: It's already "Infinite" by default In StarRocks Shared-Data (cloud-native) mode, the default behavior for any table or MV without a partition duration is to treat all data as hot data. * It will never be evicted from the disk cache based on time. * It will only be evicted if your local disk cache becomes 100% full, at which point the system uses an LRU (Least Recently Used) policy to make room for new data. 3. The "Best Way" to keep it permanently hot To ensure your dimension table is always on disk and never has to go back to S3/Iceberg, follow these two steps: Step A: Proactive Warmup (Crucial) When your MV refreshes, the new data is written to S3. It won't be on your local disks until a query asks for it. To "pin" the data to your disks immediately after every refresh, run:
Copy code
sql
CACHE SELECT * FROM default_catalog.starrocks.my_mv;
This command proactively pulls all data from the remote storage into the local disk cache of your compute nodes. Step B: Check your Cache Capacity The only way this data leaves your disk is if other queries need space. Ensure your compute nodes have enough dedicated cache space: 1. Check current usage:
SHOW BACKENDS\G
(look for
DataCacheMetrics
). 2. If your dimension tables are large, increase the
datacache_disk_size
in your
cn.conf
(compute node configuration) to ensure there is enough room for them to coexist with other query data. Summary You don't need to set any special properties. Because your table is not partitioned, StarRocks already considers its cache life "infinite." Your only task is to warm it up after each refresh using
CACHE SELECT
. ────────── Note for partitioned tables: If you ever decide to partition a table (e.g., by a
created_at
date), you would then be able to use
PROPERTIES ("datacache.partition_duration" = "100 YEAR")
at the time of creation. Referencessql-reference/sql-statements/materialized_view/ALTER_MATERIALIZED_VIEW.mdusing_starrocks/async_mv/Materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md
k
@Rocky where is documenrskrkn about CACHE SELECT *
r
The documentation for
CACHE SELECT
is located in the Block cache warmup section of the StarRocks documentation. Overview of
CACHE SELECT
CACHE SELECT
is a manual trigger used to pull data from remote storage (like S3/Iceberg) into the local disk cache of your compute nodes. This ensures that the data is "hot" and ready for high-performance queries without waiting for the first user query to fetch it. Syntax
Copy code
sql
CACHE SELECT <column_name> [, ...]
FROM [<catalog_name>.][<db_name>.]<table_name> 
[WHERE <boolean_expression>]
[PROPERTIES("verbose"="true")]
* *`*`*: You can use
*
to warm up every column in the table. *
WHERE
: You can warm up specific subsets of data (e.g., just the last 7 days of a large Iceberg table). *
PROPERTIES("verbose"="true")
: Returns detailed metrics for each individual Compute Node (CN) / Backend (BE) instead of a cluster-wide summary. Example To warm up your entire Materialized View:
Copy code
sql
CACHE SELECT * FROM default_catalog.starrocks.curated_video_mv;
Important Notes from the Docs: 1. Synchronous: The command will block your terminal/client until the data has been successfully written to the local disks. 2. Metrics: Upon completion, it returns
READ_CACHE_SIZE
(from S3) and
WRITE_CACHE_SIZE
(to local disk), allowing you to verify that the cache was populated. 3. Permissions: You must have
SELECT
privileges on the destination table to run this command. 4. Automation: You can wrap this in a scheduled task to run after every MV refresh:
Copy code
sql
    SUBMIT TASK warmup_video_mv
    SCHEDULE EVERY (INTERVAL 2 HOUR)
    AS CACHE SELECT * FROM starrocks.curated_video_mv;
Referencesdata_source/block_cache_warmup.mdusing_starrocks/caching/block_cache.mdadministration/management/resource_group.mdsql-reference/sql-statements/data-definition/CREATE_TABLE.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md Referencesdata_source/block_cache_warmup.mdusing_starrocks/caching/query_cache.mdsql-reference/sql-statements/table_bucket_part_index/SELECT.mddeployment/shared_data/feature-support-shared-data.md
k
@Rocky i do not want disk cache for some mvs to be ever evicted
r
In StarRocks, there is no specific property (like a "pin" or "no-evict" flag) that guarantees a table or Materialized View (MV) will never be evicted from the disk cache. Because the disk cache is a shared resource across all tables in a cluster, the system must be able to evict data if the physical disk becomes full to prevent the node from crashing. However, you can effectively prevent eviction for your dimension MVs by using a combination of Cache Strategy and Sizing: 1. Enable SLRU (Segmented LRU) By default, StarRocks uses a standard LRU (Least Recently Used) policy. A single large scan on a different "cold" table could potentially flush your "hot" dimension MV out of the cache. Switching to SLRU protects your hot data. It divides the cache into two segments: * Eviction Segment: Where data goes when first read. * Protection Segment: Where data is moved after it is accessed a second time. Your dimension MV, because it is queried frequently, will stay in the Protection Segment, making it much harder for a one-off query on another table to evict it. How to enable (in
cn.conf
or
be.conf
):
Copy code
properties
datacache_eviction_policy = SLRU
2. Dimension Tables are "Permanently Hot" Since your MV is a dimension table (not partitioned by time), you should not set a
partition_duration
. When this property is omitted, StarRocks treats all data as "Hot" by default. * Hot data has no expiration date. * It will only be evicted if the disk hits the
datacache_disk_high_level
(default 80% full) AND other "hot" data is being accessed more recently. 3. Use "Keep-Alive" Warmup To ensure your MV stays at the very top of the "Recently Used" list (and thus is the last thing the system would ever consider evicting), run a periodic "keep-alive" task or include it in your refresh pipeline:
Copy code
sql
-- Run this after every MV refresh
CACHE SELECT * FROM my_dimension_mv;
This forces the system to treat the entire MV as "brand new" in the cache. 4. Sizing for "No Eviction" The most reliable way to prevent eviction is to ensure your total "Must-be-Hot" data is smaller than your cache capacity: 1. Calculate the total size of your critical dimension MVs. 2. Check your cache size:
SHOW BACKENDS\G
(look for
DataCacheMetrics
). 3. Ensure
datacache_disk_size
in your config is large enough to hold all your dimension tables plus a 20-30% buffer for ad-hoc queries. Summary of Best Practices Goal | Action Prevent one-off scan eviction | Set
datacache_eviction_policy = SLRU
Prevent time-based eviction | Do not set
partition_duration
(default is infinite) Prevent LRU aging | Run
CACHE SELECT *
after every refresh Guaranteed safety | Set
datacache_disk_size
> (Total Dim Tables Size * 1.3) Referencesusing_starrocks/caching/block_cache.mddata_source/data_cache.mddata_source/block_cache_warmup.mddeployment/shared_data/feature-support-shared-data.mdadministration/management/BE_configuration.md Referencessql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md
k
@Rocky for cache select, will Starrocks attempt to refresh the entire view or will be able to refresh only parts of the view that changed based on the base table to reduce db load
r
It is important to distinguish between Materialized View (MV) Refresh (updating the data in S3/Remote Storage) and CACHE SELECT (copying that data into your local disk cache). 1. Does
CACHE SELECT
trigger an MV Refresh?
No.
CACHE SELECT
is purely a storage operation. * It reads data that already exists in your remote storage (Iceberg/S3). * It does not look at the base tables or check if the Materialized View is out of date. * If your MV is stale,
CACHE SELECT
will simply warm up the stale data. 2. Is
CACHE SELECT
incremental?
By default,
CACHE SELECT *
will scan the entire view, which can be heavy on your remote storage (I/O) and network. However, you can make it incremental by using a
WHERE
clause. This is the best way to reduce database load: * Full warmup (Heavy):
Copy code
sql
    CACHE SELECT * FROM my_mv;
* Incremental warmup (Light): If your MV is partitioned by date, you can warm up only the latest data:
Copy code
sql
    CACHE SELECT * FROM my_mv WHERE report_date = '2023-10-27';
3. Recommended Workflow for "Permanently Hot" MVs To keep your MV permanently on disk with the lowest possible load, follow this sequence: 1. Refresh the MV (Data Update): StarRocks handles this incrementally if your MV is partitioned.
Copy code
sql
    REFRESH MATERIALIZED VIEW my_mv;
2. Warm up the Cache (Performance): Use the same partition filter to pull only the new data into the disk cache.
Copy code
sql
    CACHE SELECT * FROM my_mv WHERE report_date = '2023-10-27';
Summary * StarRocks does not automatically "know" to only warm up the changed parts when you run
CACHE SELECT *
. * To reduce load, you must manually specify a
WHERE
clause in your
CACHE SELECT
statement to target the new data. * The system treats
CACHE SELECT
like a regular query (it uses the same Compute Node resources), so filtering by partition is highly recommended for large tables. Referencesdata_source/block_cache_warmup.mdusing_starrocks/caching/query_cache.mdusing_starrocks/async_mv/Materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.mdsql-reference/sql-statements/materialized_view/ALTER_MATERIALIZED_VIEW.md Referencesdata_source/block_cache_warmup.mdusing_starrocks/caching/query_cache.mddeployment/shared_data/feature-support-shared-data.md
k
@Rocky how to rename or remove scheduled automateed tasks
r
To manage scheduled automated tasks (created via
SUBMIT TASK
), use the following commands: 1. Removing a Task To delete a task, use the
DROP TASK
statement. This will also cancel any currently running instances of that task.
Copy code
sql
DROP TASK [IF EXISTS] `task_name` [FORCE];
* Recommendation: Wrap the task name in backticks (`` ` ``) to avoid syntax errors if the name contains special characters or keywords. *
FORCE
: Use this to ensure the task is removed even if there are active runs. 2. Modifying or "Renaming" a Task StarRocks does not have a direct
RENAME
command
for tasks. * To Rename: You must
DROP
the existing task and
SUBMIT
a new one with the new name. * To Update Properties: If you want to change the schedule or session variables without renaming, use `ALTER TASK`:
Copy code
sql
    -- Suspend the task
    ALTER TASK my_task SUSPEND;

    -- Resume the task
    ALTER TASK my_task RESUME;

    -- Update session properties (e.g., timeout)
    ALTER TASK my_task SET ('session.query_timeout' = '3600');
3. Finding Task Names If you aren't sure of the exact name of your automated task, you can find it by querying the information schema:
Copy code
sql
SELECT task_name, schedule, state 
FROM INFORMATION_SCHEMA.tasks;
For Materialized View refresh tasks, you can also see the associated task names using:
Copy code
sql
SHOW MATERIALIZED VIEWS;
Referencessql-reference/sql-statements/loading_unloading/ETL/DROP_TASK.mdsql-reference/sql-statements/loading_unloading/ETL/ALTER_TASK.mdsql-reference/sql-statements/loading_unloading/ETL/SUBMIT_TASK.mdsql-reference/sql-statements/materialized_view/ALTER_MATERIALIZED_VIEW.md Referencessql-reference/sql-statements/loading_unloading/ETL/ALTER_TASK.mdsql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.mdsql-reference/sql-statements/loading_unloading/ETL/DROP_TASK.mdsql-reference/sql-statements/loading_unloading/ETL/SUBMIT_TASK.md
k
@Rocky Is there a way to specify tables to never cache
r
Yes, you can specify tables to never use the Data Cache (Block Cache) by using either table-level properties or session-level variables, depending on the table type. 1. For Cloud-Native Tables and Asynchronous MVs If you are using a shared-data cluster (cloud-native tables) or Materialized Views stored in remote storage, you can disable the cache specifically for that object using the
datacache.enable
property. * At creation:
Copy code
sql
    CREATE TABLE my_table (...)
    PROPERTIES (
        "datacache.enable" = "false"
    );
* For an existing table/MV:
Copy code
sql
    ALTER TABLE my_table SET ("datacache.enable" = "false");
    ALTER MATERIALIZED VIEW my_mv SET ("datacache.enable" = "false");
When set to
false
, StarRocks will read the data directly from remote storage (S3/HDFS) and will not attempt to populate or read from the local disk cache for this specific table. 2. For External Tables (Data Lake) External tables (Hive, Iceberg, Hudi, etc.) typically use the Data Cache by default if it is enabled globally. While there isn't always a direct "per-table" property in the external catalog metadata, you can control this behavior using session variables: * Bypass cache for a specific session:
Copy code
sql
    -- This prevents the session from reading or writing to the Data Cache
    SET enable_scan_datacache = false;
    SELECT * FROM external_catalog.db.table;
* Bypass cache for a single query (using hints):
Copy code
sql
    SELECT /_+ SET_VAR(enable_scan_datacache = false) _/ * FROM external_catalog.db.table;
3. Using Partition Duration (Time-based "Never Cache") For partitioned tables, you can use
datacache.partition_duration
to ensure that data older than a certain threshold is never cached.
Copy code
sql
ALTER TABLE my_table SET ("datacache.partition_duration" = "1 DAY");
In this example, only data from the last 24 hours is eligible for the cache. Any queries scanning partitions older than 1 day will bypass the cache and read directly from remote storage. Summary of Methods Method | Scope | Best For
"datacache.enable" = "false"
| Table/MV | Permanently disabling cache for specific cloud-native data.
SET enable_scan_datacache = false
| Session/Query | One-off queries or specific ETL jobs where you don't want to "pollute" the cache with cold data.
"datacache.partition_duration"
| Table | Automatically bypassing the cache for historical/archived data. Referencesusing_starrocks/caching/block_cache.mddeployment/shared_data/s3.mddata_source/data_cache.mdsql-reference/System_variable.md Referencesusing_starrocks/caching/block_cache.mdsql-reference/System_variable.mddata_source/data_cache_troubleshooting.mddeployment/shared_data/feature-support-shared-data.mddata_source/data_cache.md
k
@Rocky does it make sense to use materialied view if im just doing a select * on an external iceberg table? or just try and load the iceberg table into data cache?
r
For a simple
SELECT *
on an external Iceberg table, the decision between a Materialized View (MV) and the Data Cache depends on your performance requirements versus maintenance effort. 1. Use Data Cache (Recommended First Step) If you are just doing a
SELECT *
without complex joins or aggregations, the Data Cache is usually the best starting point. * Why: It requires zero maintenance. StarRocks automatically caches the raw Iceberg data blocks on the local disks (SSD/NVMe) of your Compute Nodes. * Pros: No data duplication in your remote storage (S3), no refresh tasks to manage, and it automatically handles data updates (detecting changes via Iceberg metadata). * How to optimize: Use
CACHE SELECT * FROM iceberg_table
to pre-warm the cache so your first real query is already "hot." 2. Use a Materialized View (For Extreme Performance) Even for a
SELECT *
, an MV can be significantly faster than querying Iceberg with Data Cache because it transforms the data into StarRocks' internal native format. * Native Storage: StarRocks stores MV data in its own segment format, which is more highly optimized for its compute engine than generic Parquet/Iceberg files. * Indexing: You can define a Short Key Index (sorted columns) or use Z-Order on the MV. If your
SELECT *
often includes a
WHERE
clause on a specific column, a native MV will outperform Iceberg's metadata filtering. * Concurrency: MVs typically handle higher query concurrency better than external tables. Comparison Summary Feature | Data Cache (External Table) | Materialized View Maintenance | None (Automatic) | Requires Refresh Tasks Storage Cost | Low (Only local disk) | High (Full copy in S3/Remote) Performance | Fast (Reduces I/O) | Fastest (Native format + Indexes) Consistency | Stronger (Syncs with Iceberg) | Eventual (Based on refresh interval) Complexity | Simple | Moderate Recommendation * Choose Data Cache if you want a "set it and forget it" solution and your query latency is currently acceptable but you just want to speed it up. * Choose Materialized View if the Iceberg table is very large, and even with the cache, your
SELECT *
(or filtered scans) is too slow for your application's needs. Pro-Tip: If you decide to use an MV for an Iceberg table, StarRocks (v3.1.4+) supports partition-level change detection. This means the MV can incrementally refresh only the partitions that changed in Iceberg, rather than re-reading the whole table. Referencesusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.mddata_source/data_cache.mdusing_starrocks/caching/block_cache.mddata_source/catalog/iceberg_catalog.md Referencesusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.mddata_source/data_cache.mddata_source/feature-support-data-lake-analytics.mddata_source/block_cache_warmup.mdusing_starrocks/caching/block_cache.mdintroduction/StarRocks_intro.mdquick_start/iceberg.md
using_starrocks/async_mv/Materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.mdfaq/loading/Loading_faq.mdusing_starrocks/caching/query_cache.mdtable_design/Sort_key.mddata_source/catalog/iceberg_catalog.md -based_on_external_catalog_manual) • using_starrocks/async_mv/use_cases/query_acceleration_with_asynchronous_materialized_views.mdusing_starrocks/async_mv/Materialized_view_strategy.mddata_source/data_cache_troubleshooting.mdintegrations/iceberg/iceberg_guide.mdsql-reference/sql-statements/loading_unloading/ETL/SUBMIT_TASK.mdsql-reference/sql-statements/loading_unloading/ETL/CREATE_ROUTINE_LOAD.mddata_source/catalog/hive_catalog.mdusing_starrocks/async_mv/Materialized_view_concepts.mdusing_starrocks/Materialized_view.mdintegrations/loading_tools/lake2star.mdquick_start/shared_data.mdsql-reference/sql-statements/table_bucket_part_index/CREATE_TABLE.mdsql-reference/sql-statements/materialized_view/ALTER_MATERIALIZED_VIEW.mdusing_starrocks/query_acceleration_with_asynchronous_materialized_views.mdsql-reference/sql-statements/loading_unloading/ETL/DROP_TASK.mdintegrations/batch/spark.mdadministration/Query_planning.mdadministration/Configuration.mddeployment/shared_data/s3.mddeployment/shared_data/feature-support-shared-data.mdsql-reference/sql-statements/loading_unloading/ETL/ALTER_TASK.mdsql-reference/System_variable.mdsql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md
k
@Rocky how could i implement short key index and z index for a materialized view that requires these query patterns: fast lookups on a video_id column (where video_id in ...) fast lookups on a channel_id column (where channel_id in ...) fast filters on an array column (where any category is in categories column ...)
r
To optimize a Materialized View (MV) for these specific query patterns in StarRocks, you should use a combination of Z-Order clustering (for balanced multi-column filtering) and Bloom Filter indexes (for fast ID lookups). 1. Define the MV with Z-Order Clustering StarRocks (v3.3+) supports Z-Order for internal tables and MVs. Unlike a standard lexicographical sort (
ORDER BY a, b
), Z-Order ensures that filters on
video_id
or
channel_id
are both efficient, regardless of which one appears first in your query.
Copy code
sql
CREATE MATERIALIZED VIEW mv_video_stats
DISTRIBUTED BY HASH(video_id)
REFRESH ASYNC EVERY (INTERVAL 1 HOUR)
ORDER BY ZORDER(video_id, channel_id) -- Interleaved sorting for both columns
AS 
SELECT 
    video_id,
    channel_id,
    categories, -- Array column
    upload_date,
    view_count
FROM iceberg_catalog.db.raw_videos;
* Short Key Index: Because you used
ORDER BY
, StarRocks automatically creates a prefix index (Short Key) on the data. Z-Order allows the engine to skip data blocks effectively for both columns. 2. Enable Bloom Filter Indexes For
WHERE ... IN (...)
patterns, Bloom Filters are extremely effective. They allow the engine to quickly skip files that definitely do not contain the IDs you are looking for. After creating the MV, enable Bloom Filters for both ID columns:
Copy code
sql
ALTER MATERIALIZED VIEW mv_video_stats 
SET ("bloom_filter_columns" = "video_id, channel_id");
* Why?: Even if the Z-Order handles the physical layout, a Bloom Filter provides an additional "shortcut" for high-cardinality lookups like specific IDs. 3. Optimizing the Array Column Filter StarRocks does not have a dedicated "Array Index" (like a GIN index). However, you can make the
categories
filter fast using these strategies: * Rely on Pruning: Your queries should ideally filter by
video_id
or
channel_id
first. This "prunes" the data down to a few blocks, making the array scan trivial. * Query Syntax: Use
array_contains
for the best performance with the vectorized engine:
Copy code
sql
    SELECT * FROM mv_video_stats
    WHERE channel_id = 123
      AND array_contains(categories, 'Gaming');
* Advanced (Optional): If you only query by category and performance is still slow, consider creating a second MV that unnests the array. This transforms the array elements into individual rows, allowing you to use standard sorting and indexes on the category names.
Copy code
sql
    CREATE MATERIALIZED VIEW mv_categories_flattened
    ORDER BY (category_name)
    AS
    SELECT
        video_id,
        unnest as category_name
    FROM mv_video_stats, unnest(categories);
Summary of Optimizations Requirement | Solution Fast
video_id
&
channel_id
|
ORDER BY ZORDER(video_id, channel_id)
Fast
IN
/
=
Lookups
|
SET ("bloom_filter_columns" = "video_id, channel_id")
Array Filtering | Pruning via IDs + Vectorized
array_contains()
Referencestable_design/Sort_key.mdtable_design/indexes/Bloomfilter_index.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.mdsql-reference/sql-functions/array-functions/array_contains.md Referencessql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.mdtable_design/table_design.mdadministration/management/FE_configuration.md
k
@Rocky for the categories filter, the table is already 4 billion rows that will grow to 10 billion, and cateogires can contain 10 items. how practical would unest be
r
At the scale of 10 billion rows with 10 categories per row, unnesting into a single "flat" table (100 billion rows) is practical but requires a specific strategy to avoid exploding your storage and compute costs. In StarRocks, a 100-billion-row table is manageable, but you should not do a
SELECT *
unnest. Here is the recommended implementation for your scale: 1. The "Mapping Table" Strategy (Recommended) Instead of unnesting every column (which would multiply your storage by 10x), create a lightweight Materialized View that only contains the category and the primary identifier (
video_id
).
Copy code
sql
CREATE MATERIALIZED VIEW mv_category_search
PARTITION BY (upload_date) -- Match your Iceberg partition key
DISTRIBUTED BY HASH(category_name) 
ORDER BY (category_name) -- Sort by category for instant lookups
AS 
SELECT 
    video_id, 
    upload_date,
    category_name
FROM iceberg_table, unnest(categories) AS category_name;
* Why this is practical: * Incremental Refresh: StarRocks supports partition-level incremental refresh for Iceberg. When a new day of data arrives in Iceberg, StarRocks only unnests that specific day, not all 10 billion rows. * Storage Efficiency: By only including
video_id
and
category_name
, you avoid 10x storage growth for the heavy columns (titles, descriptions, metadata). * Query Rewrite: If a user runs
SELECT * FROM iceberg_table WHERE array_contains(categories, 'Gaming')
, the StarRocks optimizer can automatically join the lightweight
mv_category_search
to the base table, making the query instant. 2. For
video_id
and
channel_id
(Z-Order + Bloom Filter)
Since you need fast lookups on both IDs, do not use a standard sort key. Use Z-Order on the base table (or a separate MV if you can't touch the Iceberg layout):
Copy code
sql
-- On the base table or a "primary" MV
PROPERTIES (
    "bloom_filter_columns" = "video_id, channel_id",
    "table_sort_kind" = "ZORDER",
    "order_by" = "video_id, channel_id"
);
* Z-Order ensures that a filter on
channel_id
is just as fast as a filter on
video_id
. * Bloom Filters allow the engine to skip entire data blocks for
WHERE video_id IN (...)
queries before even reading the disk. 3. Comparison of Approaches Feature |
array_contains
(Raw)
| *Full Unnest (
SELECT *
)* | Mapping MV (Recommended) Storage | 1x (Iceberg) | ~10x (Huge) | ~1.5x (Lightweight) Performance | Slow (Full scan) | Fastest | Fast (Index + Join) Maintenance | None | High (Refresh overhead) | Moderate (Incremental) Summary Checklist for 10B+ Rows: 1. Partitioning: Ensure your MV is partitioned identically to your Iceberg table (usually by date). 2. Incremental Refresh: Use
REFRESH ASYNC
to ensure only new data is processed. 3. Pruning: If 90% of your category queries also include a
channel_id
, you might not need to unnest at all. The Z-Order on
channel_id
will prune the data enough that
array_contains
will run on a small subset in memory. 4. Inverted Index (v3.3+): If your categories were a
STRING
(e.g., "Gaming, Vlog, Tech"), you could use the new Inverted Index (built-in GIN) which is designed for this exact scale without needing to unnest rows. If you can change the Iceberg type to String, this is the most professional "modern" way to solve it. Referencesusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.mdsql-reference/sql-functions/array-functions/unnest.mdtable_design/indexes/inverted_index.mdtable_design/indexes/Bloomfilter_index.md References
k
@Rocky when fitlering for categories, i also need to select many columns
r
At the scale of 10 billion rows, unnesting "many columns" into a single wide table is impractical. It would create a 100-billion-row table, exploding your storage costs by 10x and making the
REFRESH
tasks for the Materialized View (MV) extremely slow and expensive. Instead, the "Pro" architecture for this scale involves a Primary-Secondary MV strategy. This allows you to search 100 billion category-video mappings while only storing the "many columns" once (10 billion rows). The Optimized Architecture 1. The Primary "Wide" MV (10 Billion Rows) Store all your columns here. Use Z-Order to ensure that lookups on both
video_id
and
channel_id
are equally fast.
Copy code
sql
CREATE MATERIALIZED VIEW mv_videos_wide
PARTITION BY (upload_date)
DISTRIBUTED BY HASH(video_id) -- Or channel_id, depending on your most common join
ORDER BY ZORDER(video_id, channel_id)
PROPERTIES (
    "bloom_filter_columns" = "video_id, channel_id"
)
AS 
SELECT video_id, channel_id, categories, title, description, url, metadata...
FROM iceberg_catalog.db.raw_videos;
* Performance:
video_id IN (...)
and
channel_id IN (...)
will be near-instant due to the Bloom Filter and Z-Order data skipping. 2. The Secondary "Mapping" MV (100 Billion Rows - Lightweight) Create a "Thin" index that only maps
category
to
video_id
. Since this table has only 2–3 columns, 100 billion rows will take up very little space compared to the wide table.
Copy code
sql
CREATE MATERIALIZED VIEW mv_category_index
PARTITION BY (upload_date)
DISTRIBUTED BY HASH(category_name)
ORDER BY (category_name) -- Perfect for fast equality/IN filters
AS 
SELECT 
    unnest as category_name,
    video_id,
    upload_date
FROM mv_videos_wide, unnest(categories);
How to Query Efficiently When you need to filter by category and return many columns, you join the two:
Copy code
sql
SELECT v.* 
FROM mv_videos_wide v
JOIN mv_category_index c ON v.video_id = c.video_id AND v.upload_date = c.upload_date
WHERE c.category_name = 'Gaming';
* Why this works: StarRocks' CBO (Cost-Based Optimizer) will filter the 100B mapping table first (instant, because it's sorted by
category_name
), find the matching `video_id`s, and then perform a high-speed join to the wide table to fetch the remaining columns. Alternative: The "Space-Separated String" Hack If you don't want to manage two MVs, you can use the Inverted Index (GIN) introduced in v3.3, but it requires changing the data format: 1. In your MV, store categories as a single
STRING
instead of an
ARRAY
(e.g.,
"Gaming Tech Vlog"
). 2. Create an Inverted Index on that column:
Copy code
sql
    ALTER MATERIALIZED VIEW mv_videos_wide ADD INDEX idx_cat (categories_str) USING GIN;
3. Query using `MATCH`:
Copy code
sql
    SELECT * FROM mv_videos_wide WHERE categories_str MATCH 'Gaming';
* Pros: Single table, no joins. * Cons: You lose native
ARRAY
functions (like
array_intersect
). Summary Recommendation For 10 billion rows: 1. Do NOT unnest everything. 2. Use Z-Order + Bloom Filters on your wide MV for the ID lookups. 3. Use a Mapping MV (Thin Join) for the category filter if you need to keep the
ARRAY
type, OR use the Inverted Index if you can switch to a space-separated
STRING
. Referencesusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.mdtable_design/indexes/inverted_index.mdtable_design/Sort_key.mdsql-reference/sql-functions/array-functions/unnest.md
k
@Rocky i dont have a time based column to partition by. how could i use async materialzied view instead
r
For a 10-billion-row dataset without a time-based partition column, implementing an asynchronous Materialized View (MV) requires careful architectural choices to avoid massive full refreshes and slow lookups. 1. The Partitioning Challenge (No Time Column) In StarRocks, incremental refresh for an async MV is only possible if the base table is partitioned. If your Iceberg table is not partitioned, any refresh task will attempt to rewrite all 10 billion rows, which is likely to fail or be prohibitively expensive. Solutions: * Use an Existing Column: If your base table is partitioned by something else (e.g.,
channel_id
or a
region_id
), you must use that same column in your MV's
PARTITION BY
clause to enable incremental updates. * The "Pipeline" Alternative: If your Iceberg table is truly flat (no partitions), an MV might not be the best tool. Instead, use a standard StarRocks Duplicate Key Table and manage the data loading yourself using
INSERT INTO ... SELECT
. This gives you full control over partitioning (e.g., partitioning by a hash of
video_id
). 2. Implementing Fast Lookups (video_id & channel_id) To ensure fast lookups on two different high-cardinality ID columns, use a combination of Z-Order and Bloom Filters. * Z-Order Clustering: Unlike a standard sort key (where only the first column is fast), Z-Order interleaves the data so that filters on
video_id
OR
channel_id
are both efficient. * Bloom Filter Index: This allows StarRocks to skip entire data blocks during
WHERE video_id IN (...)
lookups without even reading the disk.
Copy code
sql
CREATE MATERIALIZED VIEW mv_video_lookup
DISTRIBUTED BY HASH(video_id)
-- If base table is partitioned by 'category_id', include it here:
-- PARTITION BY category_id 
ORDER BY ZORDER(video_id, channel_id) -- Interleaved sorting
PROPERTIES (
    "bloom_filter_columns" = "video_id, channel_id"
)
AS 
SELECT video_id, channel_id, categories, title, ... 
FROM iceberg_catalog.db.raw_videos;
3. Implementing the Categories Filter (Array) At the 10-billion-row scale, filtering an
ARRAY
column with
array_contains
is slow because it requires a full scan of the column data. The "Secondary Mapping" Strategy: Since you need to return "many columns," do not unnest everything into one giant 100-billion-row table. Instead, create a thin mapping MV that only contains the category and the
video_id
. 1. Mapping MV (The Index):
Copy code
sql
    CREATE MATERIALIZED VIEW mv_category_index
    DISTRIBUTED BY HASH(category_name)
    ORDER BY (category_name) -- Perfect for fast lookups
    AS
    SELECT
        unnest as category_name,
        video_id
    FROM mv_video_lookup, unnest(categories);
2. The Query Pattern: Join the thin index back to your wide MV. The optimizer will use the
category_name
index to find the 100 Matching
video_ids
and then use the Bloom Filter/Z-Order on the wide table to fetch the rest of the columns instantly.
Copy code
sql
    SELECT v.*
    FROM mv_video_lookup v
    JOIN mv_category_index c ON v.video_id = c.video_id
    WHERE c.category_name = 'Gaming';
Summary Recommendation Requirement | Implementation Data Maintenance | Use the base table's partition column if it exists. Otherwise, consider a regular table with a manual
INSERT
pipeline. ID Lookups |
ORDER BY ZORDER(video_id, channel_id)
+
bloom_filter_columns
. Array Filtering | Create a Secondary Thin MV using
unnest
to act as a searchable index. Referencesusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md
k
@Rocky DISTRIBUTED BY HASH(video_id) how many buckets would this create? would this be efficient storage for the query use cases?
r
If you omit the
BUCKETS
keyword in
DISTRIBUTED BY HASH(video_id)
, StarRocks will use Auto-Bucketing (introduced in v2.5.7). 1. How many buckets will it create? With Auto-Bucketing, StarRocks determines the number of buckets based on your cluster's BE (Backend) count, CPU cores, and the data volume in the partition. * For a new table: It usually starts with a small number of buckets (e.g., 1 per BE or a small multiple) and increases as more data is loaded. * The Catch: For a 10-billion-row table (likely 5TB–10TB of raw data), the official recommendation is to manually set the bucket count rather than relying on auto-bucketing. * Recommendation: Aim for 10GB of raw data per tablet. * If 10 billion rows ≈ 10TB raw data, you should set
BUCKETS 1000
. * Distribute these across your BEs (e.g., if you have 10 BEs, each BE will host 100 tablets). 2. Is
HASH(video_id)
efficient for your use cases?
Your efficiency depends on which column you filter by: Use Case A: Filtering by
video_id
(Most Efficient)
* Result: Perfect. * StarRocks uses "Bucket Pruning." If you query
WHERE video_id = 123
, the engine calculates the hash and goes directly to 1 out of 1,000 buckets. It ignores 99.9% of the data. Use Case B: Filtering by
channel_id
(Inefficient for Bucketing)
* Result: Bucket Scan. * Since the data is distributed by
video_id
, a
channel_id
filter cannot prune buckets. StarRocks must scan all 1,000 buckets. * The Fix: This is why you need the Z-Order and Bloom Filter mentioned earlier. Even though the engine scans all buckets, the Bloom Filter will allow it to skip 99% of the data blocks inside those buckets. Use Case C: Filtering by
categories
(Array)
* Result: Slowest. * Bucketing by
video_id
does nothing for category filters. * The Fix: Use the Secondary Mapping MV strategy. This allows you to find `video_id`s from a category index first, then perform a "Bucket-Shuffle Join" to the main table, which is extremely fast. 3. Should you use two columns
HASH(video_id, channel_id)
?
No. If you bucket by both, a query filtering by only
video_id
will no longer be able to prune buckets. You would need both IDs in the
WHERE
clause to get pruning. Pro-Tip for 10B Rows: Since you don't have a time partition, your entire 10-billion-row dataset is technically one giant partition. 1. Manual Buckets: Use
DISTRIBUTED BY HASH(video_id) BUCKETS 1000
(adjust based on your actual data size). 2. Avoid Tablet Explosion: Don't go over 2,000 buckets unless your cluster is massive (50+ BE nodes), as too many tablets increase metadata pressure on the FE. Referencestable_design/data_distribution/Data_distribution.mdfaq/Deploy_faq.mdtable_design/indexes/Bloomfilter_index.mdadministration/management/Scale_up_down.md Referencestable_design/data_distribution/Data_distribution.mdfaq/Deploy_faq.mdtable_design/Sort_key.mddeployment/plan_cluster.md Referencestable_design/data_distribution/Data_distribution.md
k
@Rocky i am on starrocks 4.1.1 and ZORDER is not working Query 1 ERROR at Line 66: : Getting syntax error at line 3, column 9. Detail message: Unexpected input 'ZORDER', the most similar input is {'('}.
r
The error occurred because
ZORDER
is not a valid SQL keyword or function in StarRocks
. While Z-Order is a popular multi-column clustering technique in some other engines (like Databricks or Delta Lake), StarRocks achieves similar multi-column filtering performance through its native Prefix Index and Sort Key architecture. To fix your syntax error and achieve high performance for your 10-billion-row table, use the following StarRocks-native strategies: 1. Correct the Syntax: Use a Standard Sort Key The
ORDER BY
clause in a StarRocks
CREATE TABLE
or
CREATE MATERIALIZED VIEW
statement defines the Sort Key. For a wide table where you filter by both
video_id
and
channel_id
, you should list them in order of importance:
Copy code
sql
CREATE MATERIALIZED VIEW mv_videos_wide
DISTRIBUTED BY HASH(video_id) -- or HASH(channel_id)
ORDER BY (video_id, channel_id) -- Use parentheses, no ZORDER keyword
PROPERTIES (
    "bloom_filter_columns" = "video_id, channel_id"
)
AS 
SELECT video_id, channel_id, ... 
FROM base_table;
2. How to achieve "Z-Order-like" performance Because StarRocks uses a lexicographical sort (it sorts by the first column first, then the second), the first column in
ORDER BY
gets the best pruning (near-instant lookups). To make the second column (
channel_id
) just as fast, you must use the
bloom_filter_columns
property. * Primary Filter (
video_id
):
Uses the Prefix Index (extremely fast binary search). * Secondary Filter (
channel_id
):
Uses the Bloom Filter (allows the engine to skip ~99% of data blocks without reading them from disk). 3. Use an Inverted Index (StarRocks v3.3+) If you need to select many columns and filter by multiple high-cardinality IDs (or categories) without worrying about column order, the Inverted Index is the most powerful tool in StarRocks 4.1.1. It allows "index-only" filtering on any column:
Copy code
sql
-- After creating the MV, add inverted indexes
ALTER MATERIALIZED VIEW mv_videos_wide 
ADD INDEX idx_vid (video_id) USING GIN;

ALTER MATERIALIZED VIEW mv_videos_wide 
ADD INDEX idx_cid (channel_id) USING GIN;
This is often faster than a Z-Order curve for random-access lookups on 10B rows. 4. Special Case:
encode_sort_key
(StarRocks 4.0+)
If you are trying to create a composite key for a very specific ordering requirement, StarRocks 4.0 introduced the
encode_sort_key()
function. It creates a binary composite key from multiple columns that can be used in your
ORDER BY
clause:
Copy code
sql
CREATE TABLE example (
    v1 INT,
    v2 VARCHAR,
    -- Generate a composite sort key
    sort_col VARBINARY(1024) AS (encode_sort_key(v1, v2))
)
ORDER BY (sort_col);
(Note: This is still a lexicographical composite, not a bit-interleaved Z-Order curve, but it is the official way to handle complex multi-column sorting.) Summary for your 4.1.1 Setup: 1. Remove
ZORDER(...)
from your SQL; it is causing the syntax error. 2. Use
ORDER BY (video_id, channel_id)
for the primary sort. 3. Add
"bloom_filter_columns" = "video_id, channel_id"
to handle the "non-prefix" lookups. 4. Consider an Inverted Index (GIN) for even faster filtering on the category columns. Referencestable_design/indexes/Prefix_index_sort_key.mdtable_design/indexes/Bloomfilter_index.mdtable_design/indexes/inverted_index.mdsql-reference/sql-functions/utility-functions/encode_sort_key.md Referencessql-reference/sql-functions/utility-functions/encode_sort_key.mdbest_practices/table_clustering.md
k
@Rocky if i dont pass video_id to filter, isnt channel_id filter useless
r
Strictly speaking, the
channel_id
filter is not useless, but it is significantly less efficient than the
video_id
filter if you rely solely on the Sort Key (Prefix Index). Here is the breakdown of how StarRocks handles a query that filters on the second column (
channel_id
) without the first (
video_id
): 1. The Prefix Index Limitation StarRocks builds a sparse index based on the first ~36 bytes of your
ORDER BY
columns. * With `video_id`: The engine uses binary search to jump straight to the matching rows. * Without `video_id`: The Prefix Index cannot be used. The engine cannot "jump" because it doesn't know where the
channel_id
values are located across the global sort order of
video_id
. 2. Why it’s NOT Useless (Zone Map Pruning) Even if the Prefix Index fails, StarRocks has a second layer of defense: Zone Maps. StarRocks stores the
min/max
values for every column in every 64KB data page and every segment file. * Because the data is sorted by
(video_id, channel_id)
, the
channel_id
values are "clustered" within each
video_id
. * While not as powerful as a prefix search, the storage engine can still use these min/max values to skip segments or pages where your target
channel_id
definitely doesn't exist. This is still much faster than a "full table scan" in a traditional database. 3. How to make
channel_id
just as fast
To make
channel_id
highly efficient without needing
video_id
, you have two primary tools in StarRocks 4.1.1: A. Bloom Filter Index (Best for
=
and
IN
)
Adding a Bloom Filter allows StarRocks to check a small bit-map to see if a
channel_id
exists in a data block before reading it.
Copy code
sql
ALTER MATERIALIZED VIEW mv_name SET ("bloom_filter_columns" = "channel_id");
* Performance: Very high for point lookups. It allows the engine to skip ~99% of data blocks even if they aren't pruned by the Sort Key. B. Inverted Index (Best for 10B+ Rows) In StarRocks 4.1.1, the Inverted Index (GIN) is the most robust solution for non-prefix columns. It creates a separate index structure that maps values to row IDs.
Copy code
sql
ALTER MATERIALIZED VIEW mv_name ADD INDEX idx_channel (channel_id) USING GIN;
* Performance: This effectively makes the column order in
ORDER BY
irrelevant for filtering. You can filter by
channel_id
alone, and the engine will use the GIN index to find the rows instantly. Summary Filter used | Sort Key (Prefix) | Zone Map Pruning | Bloom Filter / GIN
video_id
| Yes (Instant) | Yes | Optional
channel_id
| No | Partial (Clustered) | Highly Recommended Recommendation: Keep
video_id
as the first column in your
ORDER BY
if it's your most common filter, but add an Inverted Index or Bloom Filter on
channel_id
. This gives you "best of both worlds" performance on a 10-billion-row dataset. Referencestable_design/indexes/Prefix_index_sort_key.mdtable_design/indexes/Bloomfilter_index.mdtable_design/indexes/inverted_index.mdbest_practices/table_clustering.md Referencesknowledge/usage/best-practice.mdtable_design/indexes/Prefix_index_sort_key.mdbest_practices/table_clustering.mdfaq/Sql_faq.md
k
@Rocky uery 1 ERROR at Line 47: : Getting analyzing error. Detail message: The inverted index is disabled, enable it by setting FE config
enable_experimental_gin
to true.
r
In StarRocks 4.1.1, the Inverted Index (GIN) is currently a beta feature and is disabled by default. You need to enable it at the Frontend (FE) level to use it. 1. Enable it Dynamically (Immediate Effect) Run the following command in your SQL client (requires
SYSTEM
level privileges):
Copy code
sql
ADMIN SET FRONTEND CONFIG ("enable_experimental_gin" = "true");
2. Enable it Permanently (Survival after Restart) The
ADMIN SET
command above only lasts until the FE service restarts. To make the change permanent, you must modify the configuration file on every FE node: 1. Locate your
fe.conf
file (usually in the
fe/conf/
directory). 2. Add or update the following line:
Copy code
conf
    enable_experimental_gin = true
3. Restart the FE service for the file change to take effect. Important Notes for Version 4.1.1 * Built-in Implementation: Since you are on version 4.1.1, you can use the new built-in inverted index instead of the default CLucene-based one. This is recommended if you are on a shared-data (cloud-native) cluster or if you want better integration with StarRocks' native bitmap structures. * To use it, add
"imp_lib" = "builtin"
to your index properties:
Copy code
sql
        INDEX idx_name (column_name) USING GIN("imp_lib" = "builtin");
* Replicated Storage: When using Inverted Indexes, the
replicated_storage
property for the table must be
false
. In version 4.1.1, StarRocks usually handles this automatically when the index is created, but it's good to keep in mind if you see errors regarding storage type. Referencestable_design/indexes/inverted_index.mdsql-reference/sql-statements/cluster-management/config_vars/ADMIN_SET_CONFIG.mdadministration/management/FE_configuration.md Referencestable_design/indexes/inverted_index.md
k
@Rocky i am using starrocks for dimensional data which involves full row based access. can i change the data format of a mv to be more row based to reduce the cost of projecting many columns
r
Yes, you can optimize for full-row access in StarRocks 4.1.1 by using Hybrid Row-Column Storage. By default, StarRocks is a columnar database, which is expensive for "point lookups" or "wide-row projections" because it must fetch data from many separate files (one per column). Hybrid storage solves this by storing each row as a single contiguous binary block (an internal
__row
column) alongside the columnar data. 1. Requirements * Primary Key Model: Hybrid storage is only supported for Primary Key tables. * Asynchronous Materialized Views: Since version 3.3, StarRocks allows you to define a Primary Key for an asynchronous MV, making it eligible for hybrid storage. * Shared-data Clusters: Currently, hybrid storage is not supported in StarRocks shared-data (cloud-native) mode; it is for shared-nothing clusters only. 2. How to Enable it First, you must enable the experimental feature at the FE level:
Copy code
sql
ADMIN SET FRONTEND CONFIG ("enable_experimental_rowstore" = "true");
3. Create the MV with Hybrid Storage When creating your Materialized View, you must explicitly define the
PRIMARY KEY
and set the
store_type
property to `column_with_row`:
Copy code
sql
CREATE MATERIALIZED VIEW mv_dimensional_data
PRIMARY KEY (id)
DISTRIBUTED BY HASH(id)
PROPERTIES (
    "store_type" = "column_with_row"
)
AS 
SELECT 
    id, 
    attr1, 
    attr2, 
    attr3, 
    ... -- project many columns
FROM base_table;
How it works: * Standard OLAP: Queries that aggregate data (e.g.,
SUM(revenue)
) still use the efficient columnar format. * Full Row Access: Queries that fetch many columns for a specific ID (e.g.,
SELECT * FROM mv WHERE id = 123
) will trigger a "short-circuit" read. StarRocks will fetch the pre-packaged binary row from the
__row
column in one I/O operation instead of merging dozens of columns. Considerations: * Storage Overhead: This will roughly double the storage space used by the MV, as data is stored in both column and row formats. * Row Size Limit: The total size of a single row cannot exceed 1 MB. * Refresh Performance: Enabling row storage can slightly increase the time it takes for the MV to refresh, as it has to encode the binary row during the write phase. Referencestable_design/hybrid_table.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.mdadministration/management/FE_configuration.md Referencestable_design/hybrid_table.mdadministration/management/FE_configuration.mdusing_starrocks/async_mv/Materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.mdtable_design/hybrid_table.mdadministration/management/FE_configuration.mdusing_starrocks/async_mv/Materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md Referencessql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md
k
@Rocky how to starrocks grant select on tables with pattern "GRANT SELECT ON TABLE 'abc.def*' to 'user'"
r
StarRocks does not support using wildcard patterns (like
_
or
%
) for partial table names in the
GRANT
statement. The only wildcards supported are
_._
(all databases and all tables) or
db_name._
(all tables within a specific database). To grant privileges on a subset of tables matching a pattern (e.g., all tables starting with
def
in database
abc
), you must use one of the following two approaches: Option 1: Generate GRANT statements via SQL (Recommended) You can query the
information_schema.tables
metadata to generate a list of
GRANT
statements for all tables that match your pattern. Run this query to generate the commands:
Copy code
sql
SELECT CONCAT('GRANT SELECT ON TABLE ', table_schema, '.', table_name, " TO 'user'@'%';")
FROM information_schema.tables
WHERE table_schema = 'abc' 
  AND table_name LIKE 'def%';
Then, copy the output and execute the resulting
GRANT
statements. Option 2: Use Comma-Separated List If you have a small number of tables, StarRocks supports granting privileges on multiple specific tables in a single statement by separating them with commas (supported in recent versions):
Copy code
sql
GRANT SELECT ON TABLE abc.def_1, abc.def_2, abc.def_v3 TO USER 'user'@'%';
Important Syntax Rules * Database-level:
GRANT SELECT ON abc.* TO USER 'user'@'%';
(Grants to all tables in
abc
). * Specific Tables: Requires the
TABLE
keyword:
GRANT SELECT ON TABLE abc.def_1 TO USER 'user'@'%';
. * Case Sensitivity: Table names in the pattern match (
LIKE 'def%'
) are generally case-sensitive depending on your
lower_case_table_names
configuration. Referencessql-reference/sql-statements/account-management/GRANT.mdadministration/user_privs/authorization/User_privilege.md Referencessql-reference/sql-statements/account-management/GRANT.mdfaq/operation_maintenance_faq.mddata_source/catalog/hive_catalog.mdadministration/user_privs/authorization/User_privilege.md
k
@Rocky how do i view my scheudled tasks that i created with SUBMIT TASK
r
To view the scheduled tasks you created using the
SUBMIT TASK
command, you should query the system metadata views in the
information_schema
. 1. View Task Definitions and Schedules To see a list of all submitted tasks, their SQL definitions, and their schedules (if periodic), query the
tasks
view:
Copy code
sql
SELECT * FROM information_schema.tasks;
Common filters: * Find a specific task by name:
Copy code
sql
    SELECT * FROM information_schema.tasks WHERE task_name = 'your_task_name';
* See the SQL definition for a task:
Copy code
sql
    SELECT definition FROM information_schema.tasks WHERE task_name = 'your_task_name';
2. View Task Execution History (Runs) Each time a scheduled task triggers (or when a one-time task executes), it creates a "Task Run." To check the status (SUCCESS, FAILED, RUNNING) or the progress of these executions, query the
task_runs
view:
Copy code
sql
SELECT * FROM information_schema.task_runs;
Common filters: * Check the status of the most recent runs for a task:
Copy code
sql
    SELECT task_name, status, create_time, finish_time, error_message
    FROM information_schema.task_runs
    WHERE task_name = 'your_task_name'
    ORDER BY create_time DESC;
Summary of Views View | Purpose
information_schema.tasks
| Shows the "template" or job definition (Name, Schedule, SQL).
information_schema.task_runs
| Shows the actual execution instances and their outcomes. Related Commands * Stop/Delete a task:
DROP TASK <task_name>;
* Modify a task:
ALTER TASK <task_name> SET SCHEDULE EVERY (INTERVAL 1 DAY);
* Manually trigger a task immediately:
INSERT INTO TABLE task_runs (task_name) VALUES ('your_task_name');
(Note: In some versions, you may need to use
SUBMIT TASK
again for manual triggers). Referencessql-reference/information_schema/tasks.mdsql-reference/sql-statements/loading_unloading/ETL/SUBMIT_TASK.mdsql-reference/information_schema/task_runs.md Referencessql-reference/information_schema/tasks.mdsql-reference/sql-statements/loading_unloading/ETL/SUBMIT_TASK.mdsql-reference/sql-statements/loading_unloading/routine_load/SHOW_ROUTINE_LOAD_TASK.md
k
@Rocky i have an iceberg table partition: PARTITION BY month(
date
) how do i partition by the same scheme with async partitioned mv
r
To partition an asynchronous materialized view (MV) in StarRocks to match an Iceberg table partitioned by
month(date)
, you should use the
date_trunc
function in your MV's
PARTITION BY
clause. Starting from StarRocks v3.2.3, StarRocks supports aligning MV partitions with Iceberg "Partition Transforms." The Mapping To align with an Iceberg
month
transform, use the following syntax: * Iceberg Partition:
month(date_col)
* StarRocks MV Partition:
PARTITION BY date_trunc('month', date_col)
Example If your base Iceberg table is
iceberg_catalog.db.orders
and is partitioned by
month(order_date)
, create your MV like this:
Copy code
sql
CREATE MATERIALIZED VIEW mv_monthly_orders
PARTITION BY date_trunc('month', order_date)
REFRESH ASYNC
AS 
SELECT 
    order_date,
    order_id,
    amount
FROM iceberg_catalog.db.orders;
Key Details 1. Version Support: This feature was introduced in v3.2.3. For multi-column partition alignment, v3.5.0+ is recommended. 2. Incremental Refresh: Because the partitions are aligned, StarRocks can detect which specific months have changed in the Iceberg table and only refresh those corresponding partitions in the MV, rather than performing a full refresh. 3. Column Type: Ensure the
order_date
column in the
SELECT
list matches the source column type (usually
DATE
or
DATETIME
). 4. Automatic Mapping: You do not need to create a manual string-based partition;
date_trunc
tells StarRocks to group the data into monthly buckets that match Iceberg's internal structure. Iceberg Transform | StarRocks MV Partition Expression
year(col)
|
date_trunc('year', col)
month(col)
|
date_trunc('month', col)
day(col)
|
date_trunc('day', col)
hour(col)
|
date_trunc('hour', col)
Referencesusing_starrocks/async_mv/use_cases/create_partitioned_materialized_view.mdusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md Referencesusing_starrocks/async_mv/use_cases/create_partitioned_materialized_view.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.mdusing_starrocks/async_mv/use_cases/data_lake_query_acceleration_with_materialized_views.md
k
@Rocky in a shared data cluster, with data cache sharing how much slow will reads be for a new spot compute node compared to a cached on disk mv read?
r
In a StarRocks shared-data cluster, the read performance for a new spot compute node depends significantly on whether Data Cache Sharing is enabled and how much data is being fetched. Here is the breakdown of the expected slowdown for a new node with an empty local cache: 1. With Data Cache Sharing Enabled If you have enabled Data Cache Sharing (
enable_datacache_sharing = true
), the new node will attempt to fetch data from the local disks of other warm nodes in the cluster over the internal network (VPC). * Estimated Slowdown: ~1.1x to 1.5x slower than a local cache hit. * Why: Modern cloud networks (10Gbps–100Gbps) have very low latency (sub-millisecond). Fetching a block of data from a peer node's NVMe disk over the network is slightly slower than reading from a local disk but is significantly faster than going to object storage. * Observability: You can monitor this in the Query Profile using the metric
DataCacheReadPeerBytes
. 2. Without Data Cache Sharing (Pure Remote Read) If Data Cache Sharing is disabled or if the data doesn't exist on any other node's cache, the new node must fetch data directly from remote storage (e.g., S3, GCS, or HDFS). * Estimated Slowdown: ~2.5x to 3x slower than a local cache hit. * Why: Official StarRocks SSB benchmarks (1TB dataset) show that a shared-data cluster with no cache (remote reads only) takes roughly 2.8 times longer to complete queries than one with a warm local cache. * Performance Jitter: You will notice high
IOTimeRemote
and
CompressedBytesReadRemote
in your profile. Summary Table: Read Performance Comparison Scenario | Data Source | Typical Latency | Estimated Slowdown Warm Node (Local Hit) | Local NVMe/SSD | < 1ms | 1.0x (Baseline) New Node (Peer Hit) | Peer Node Disk via Network | 1ms - 5ms | ~1.2x New Node (Cold Read) | Remote Storage (S3/GCS) | 50ms - 200ms | ~2.8x Recommendations for Spot Instances To minimize the impact of adding new spot nodes: 1. Enable Cache Sharing: This is the most effective way to prevent "cold start" latency.
Copy code
sql
    -- Enable on FE
    ADMIN SET FRONTEND CONFIG ("enable_trace_historical_node" = "true");
    -- Enable for the session or globally
    SET GLOBAL enable_datacache_sharing = true;
2. Use Data Cache Warmup: If you know a specific Materialized View or table will be queried heavily after scaling, use the
CACHE SELECT
command to proactively pull that data into the cache of the new nodes. 3. Monitor Convergence: As the spot node executes queries, it will save the blocks it fetches to its own local disk. Its performance will naturally improve until it matches a warm node (usually after 1-2 runs of the same query). Referencesdata_source/data_cache.mddeployment/shared_data/feature-support-shared-data.mdfaq/shared_data_faq.md Referencesfaq/shared_data_faq.mddeployment/shared_data/feature-support-shared-data.mddata_source/data_cache.md
k
@Rocky some of my queries are hitting bottlenecks for cpu contention from a resource group, where some queries are taking 30 sec that should take < 1 sec. What are some cluster configurations I can tune to increase qps and reduce bottleneck? Queries from this app are already in a dedicated resource group with 60 exclusive cpu % The other 40% is left for other app queries like etl and motoring
r
To resolve CPU contention and reduce the "30-second stall" issue for high-concurrency apps, you should focus on three areas: reducing execution overhead (DOP), managing entry into the cluster (Queuing), and refining the isolation strategy (Hard vs. Soft limits). 1. Optimize
pipeline_dop
for High QPS
High QPS environments with short queries often suffer from thread context switching and scheduling overhead. If a query takes < 1 second, it does not need high parallelism. * The Issue: If
pipeline_dop
is high (e.g., set to the number of cores), a single query spawns dozens of "drivers." If you run 50 concurrent queries, you could have thousands of drivers fighting for those 60% cores. * The Fix: Set a lower
pipeline_dop
for this specific app's session or resource group.
Copy code
sql
    -- For short, high-frequency queries, 1, 2, or 4 is often optimal.
    SET GLOBAL pipeline_dop = 2;
2. Configure Resource Group-Level Queuing If queries are taking 30 seconds, they might be waiting for execution slots or thrashing the CPU. You should enable the Query Queue at the resource group level to prevent the system from being overwhelmed. * The Fix: Add a
concurrency_limit
to your resource group. Instead of letting 100 queries try to run at once (and all take 30s), let 20 run at full speed (<1s) and queue the rest.
Copy code
sql
    ALTER RESOURCE GROUP your_app_group WITH (
        "concurrency_limit" = "20" -- Adjust based on your core count and testing
    );
    -- Ensure the queue is enabled
    SET GLOBAL enable_group_level_query_queue = true;
* Monitor: Check the
PendingTimeMs
field in your
fe.audit.log
. If this value is high, queries are safely waiting in the queue rather than competing for CPU. 3. Switch to
cpu_weight
(Soft Isolation)
You mentioned "60% exclusive cpu." In StarRocks,
exclusive_cpu_cores
is a hard limit. * The Problem: Your app cannot use the other 40% of the CPU even if the ETL/monitoring tasks are idle. * The Fix: Use
cpu_weight
instead. This provides soft isolation. * If the cluster is fully loaded, your app is guaranteed 60% of the CPU. * If the ETL (40%) is idle, your app can automatically scale up to 100% of the CPU, significantly improving latency during spikes.
Copy code
sql
    -- Remove exclusive cores and set weight instead (e.g., 60 vs 40)
    ALTER RESOURCE GROUP your_app_group WITH (
        "exclusive_cpu_cores" = "0",
        "cpu_weight" = "60"
    );
4. Use "Big Query" Protection If your app occasionally sends a "heavy" query (e.g., a large export or a wide-range scan), it could be consuming all 60% of the CPU, starving the small < 1s queries within the same group. * The Fix: Set limits to automatically kill or restrict "runaway" queries within the group:
Copy code
sql
    ALTER RESOURCE GROUP your_app_group WITH (
        "big_query_cpu_second_limit" = "10", -- Kill query if it uses > 10s of CPU on a single BE
        "big_query_mem_limit" = "2147483648" -- 2GB limit per query on a BE
    );
5. Check Internal Driver Limits If you are on version v3.1.4+, check the dynamic driver watermarks. These variables prevent the pipeline engine from being flooded: * `query_queue_driver_low_water`: The threshold where StarRocks begins reducing
pipeline_dop
for new queries. * `query_queue_driver_high_water`: The absolute cap for concurrent drivers. You can view current load via
SHOW PROC '/backends';
and looking at the
NumRunningQueries
and
CpuUsedPct
columns. Referencesadministration/management/resource_management/query_queues.mdadministration/management/resource_management/resource_group.mdfaq/resource_isolation_faq.md References
k
@Rocky what causes this: Warehouse default_warehouse is not available.
r
The error
Warehouse default_warehouse is not available
typically occurs in StarRocks Shared-Data (Cloud-Native) clusters. It indicates that the Frontend (FE) cannot find or connect to any active compute resources assigned to the default compute group (the warehouse). Here are the most common causes and how to fix them: 1. No Active Compute Nodes (CN) The most frequent cause is that all Compute Nodes (CNs) assigned to the
default_warehouse
are offline or have not been started. * Check Status: Run the following command in your MySQL client:
Copy code
sql
    SHOW COMPUTE NODES;
    -- or if using multiple warehouses
    SHOW WAREHOUSES;
* Solution: Ensure your CN processes are running. If you are using Kubernetes, check if the CN pods are in a
Running
state. If they are down, check the
cn.out
or
<http://be.INFO|be.INFO>
logs on the nodes for crash details. 2. Backends/CNs Not Added to the Warehouse In a shared-data architecture, compute nodes must be explicitly added to the FE. If the FE has started but no nodes have been registered, the
default_warehouse
exists in name only but has no "availability." * Solution: Add the compute nodes manually if they aren't showing up:
Copy code
sql
    ALTER SYSTEM ADD COMPUTE NODE "host:port";
3. Metadata Inconsistency (FE Restart/Upgrade) If you recently restarted your cluster or performed an upgrade, there may be a temporary metadata mismatch where the FE thinks the warehouse is in a "SUSPENDED" or "INITIALIZING" state. * Solution: Try to manually resume or "touch" the warehouse:
Copy code
sql
    ALTER WAREHOUSE default_warehouse RESUME;
4. Connection Issues between FE and CN The FE might see the warehouse, but it cannot communicate with the CNs due to networking issues (firewalls, incorrect IP/FQDN, or port mismatches). * Check: Verify that the FE can reach the CNs on the
heartbeat_service_port
(default
9050
) and
brpc_port
(default
8060
). * Logs: Look for
Failed to send heartbeat
or
RPC call failed
errors in the
fe.log
. 5. Cloud-Native/Enterprise Feature Lock The "Warehouse" syntax and management are primary features of StarRocks v3.0+ Shared-Data and the Enterprise Edition. If you are on a Community Edition (Shared-Nothing) cluster and seeing this error, it usually means you are accidentally using shared-data commands or a session variable has been set that expects a warehouse architecture (e.g.,
SET WAREHOUSE = ...
). Quick Debug Checklist: 1. Run
SELECT current_warehouse();
to confirm you are indeed targeting
default_warehouse
. 2. Run
SHOW BACKENDS;
or
SHOW COMPUTE NODES;
. If the list is empty or
Alive
is
false
, your compute layer is down. 3. Check FE Logs: Search
fe.log
for the string
WarehouseManager
to see if there were errors initializing the default warehouse during startup. Referencessql-reference/sql-functions/utility-functions/current_warehouse.mdquick_start/shared-data.mdfaq/shared_data_faq.mddata_source/catalog/default_catalog.mdsql-reference/information_schema/warehouse_queries.mdfaq/resource_isolation_faq.mdsql-reference/information_schema/warehouse_metrics.mdsql-reference/sql-statements/cluster-management/storage_volume/SET_DEFAULT_STORAGE_VOLUME.md
k
@Rocky sql gui works but an application is experiencing this error
r
If your SQL GUI (like DBeaver or MySQL Workbench) works but your application fails with "Warehouse default_warehouse is not available," the issue is almost certainly due to user permissions or unactivated roles for the specific user account your application uses. In StarRocks v3.x (Shared-Data architecture), compute resources are managed via Warehouses. To run a query, a user must have the
USAGE
privilege on the warehouse being used. 1. Grant Warehouse Usage to the App User If your application connects with a dedicated user (e.g.,
app_user
), it likely lacks the permission to use the
default_warehouse
. Run this command as a
root
or
admin
user:
Copy code
sql
-- Replace 'app_user' with your actual application username
GRANT USAGE ON WAREHOUSE default_warehouse TO USER 'app_user'@'%';
2. Activate Roles on Login Even if you granted the permission via a role, StarRocks does not always activate all roles by default when a session starts. If the GUI works (likely as
root
), but the app fails, the app's user might have the role but it isn't "active." You can force all roles to be active for everyone upon login:
Copy code
sql
SET GLOBAL activate_all_roles_on_login = TRUE;
Or set a default role for that specific user:
Copy code
sql
ALTER USER 'app_user'@'%' DEFAULT ROLE 'your_app_role';
3. Check for Connection String Overrides Some application frameworks or BI tools (like Metabase or Superset) allow you to specify a "Warehouse" or "Namespace" in the connection settings. * If your app is explicitly trying to use a warehouse name that doesn't exist (or one that the user can't see), it may default back to a "not available" state. * Ensure your JDBC/Python connection string is not passing an incorrect warehouse parameter. 4. Verify the App User's Context If you can run a "raw" SQL query through your application's connection, try running these to see the difference between the GUI and the App:
Copy code
sql
SELECT current_warehouse(); -- Should return 'default_warehouse'
SELECT current_role();      -- See which roles are actually active for the app
SHOW GRANTS;                -- Verify the user has USAGE on the warehouse
5. Backend/CN Connectivity (Less Likely if GUI works) If the above steps don't work, ensure your application is hitting the same Frontend (FE) IP as your GUI. If you have multiple FEs and one has a metadata sync issue, it might report the warehouse as unavailable while another FE sees it as fine. Referencessql-reference/sql-functions/utility-functions/current_warehouse.mdadministration/user_privs/authorization/privilege_faq.mdadministration/user_privs/authorization/User_privilege.md Referencesadministration/user_privs/authorization/built_in_roles.mdadministration/user_privs/authorization/user_privs.mdadministration/user_privs/authorization/User_privilege.mdadministration/user_privs/authorization/privilege_faq.md
k
@Rocky select current_warehouse is working, but a select query against a table is not
r
If
SELECT current_warehouse()
works but a table query fails with "Warehouse not available," it confirms that your Frontend (FE) is healthy, but it cannot find any active compute resources to actually run the query.
current_warehouse()
is a metadata function that runs entirely on the FE. A
SELECT
from a table, however, requires the FE to dispatch tasks to a Compute Node (CN) or Backend (BE). Follow these steps to find the break in the chain: 1. Check if the Compute Nodes are "Alive" The FE might know about the nodes, but they may be offline or failing heartbeats. Run this command:
Copy code
sql
SHOW COMPUTE NODES;
-- If you are not using Shared-Data architecture, use:
SHOW BACKENDS;
* What to look for: Look at the
Alive
column. If it says
false
, the compute process is down. * Action: Restart the CN or BE processes on your worker nodes and check their
<http://be.INFO|be.INFO>
or
<http://cn.INFO|cn.INFO>
logs for errors. 2. Verify Nodes are Assigned to the Warehouse A warehouse can exist as a name, but it might have zero nodes assigned to it. Run this command:
Copy code
sql
SHOW WAREHOUSES;
* What to look for: Check the node count or status for
default_warehouse
. * Action: If the warehouse is empty, you need to add nodes to it or ensure your nodes are configured to join the
default_warehouse
group. 3. Check for "Suspended" State (Shared-Data) If you are on a Cloud-Native (Shared-Data) cluster, warehouses can be suspended to save costs. Usually, they auto-resume, but this can fail if auto-resume is disabled or if there's a permission issue. * Action: Try to manually resume it:
Copy code
sql
    ALTER WAREHOUSE default_warehouse RESUME;
4. Verify "USAGE" Privilege If your application user can see the warehouse name but the query fails, they might lack the USAGE privilege. In StarRocks, simply "knowing" a warehouse exists isn't enough; the user must be authorized to use it for computation. * Action (Run as admin):
Copy code
sql
    GRANT USAGE ON WAREHOUSE default_warehouse TO USER 'your_app_user'@'%';
5. Check FE-to-CN Connectivity The FE may think the node is alive, but fails when it tries to send the query plan via RPC (Remote Procedure Call). * Action: Check your
fe.log
. Look for errors like
RPC failed
or
Master find no backend
. * Common Culprit: A firewall or security group blocking the brpc_port (default
8060
) or heartbeat_service_port (default
9050
) between the FE and the CN/BE nodes. Summary Checklist: 1. Is the CN/BE process running? (
ps aux | grep starrocks
) 2. Does
SHOW COMPUTE NODES
show
Alive: true
?
3. Does the app user have
USAGE
rights on the warehouse?
Referencesfaq/shared_data_faq.mdquick_start/shared-data.mddeployment/shared_data/hdfs.mddeployment/shared_data/gcs.mddeployment/shared_data/s3.md
k
@Rocky this is only happening from a specific application with a valid connection string
r
If
SELECT current_warehouse()
works (returning
default_warehouse
) but a table query fails, it means your application has a valid session context, but the execution layer cannot find or reach the compute nodes for that specific session. Since your SQL GUI works but the application fails, the problem is likely one of the following: 1. Missing
USAGE
Privilege or Role Activation
Even if a user has a "default warehouse" assigned, they cannot execute queries on it without the
USAGE
privilege. * The Symptom:
current_warehouse()
reads from session metadata (works), but a
SELECT
triggers a compute check (fails). * The Fix: Ensure the application user has the correct permissions and that their roles are active. Run this as an admin:
Copy code
sql
    -- Grant usage if not already present
    GRANT USAGE ON WAREHOUSE default_warehouse TO USER 'your_app_user'@'%';

    -- Ensure roles are automatically activated for the application session
    SET GLOBAL activate_all_roles_on_login = TRUE;
2. Warehouse Suspension & Auto-Resume Permissions In StarRocks Shared-Data, warehouses can enter a
SUSPENDED
state to save costs. * The Problem: Your GUI might be logged in as
root
or an admin who has permission to auto-resume the warehouse. Your application user might lack this permission. When the app runs a query, the warehouse stays suspended, and the FE reports it as "not available." * The Fix: Check the warehouse status:
SHOW WAREHOUSES;
. If it is suspended, try to manually resume it as the app user. If that fails, grant the resume privilege:
Copy code
sql
    GRANT ALTER ON WAREHOUSE default_warehouse TO USER 'your_app_user'@'%';
3. Load Balancer / Multiple Frontends (FE) If you have multiple FEs behind a load balancer, your GUI might be hitting FE-A (which is healthy), while your application is hitting FE-B (which might have a metadata sync issue or cannot "see" the compute nodes). * The Test: Point your application's connection string directly to the same FE IP address that your GUI uses. If it starts working, the issue is a specific FE node or the load balancer configuration. 4. Connection String / JDBC Parameters Check if your application's connection string includes a
warehouse
parameter or if the application framework (like Spring, Hibernate, or a Python ORM) is executing an initialization SQL. * Check for Casing: StarRocks warehouse names can be case-sensitive depending on your configuration. Ensure the app isn't trying to set
default_warehouse
if the actual name is
Default_Warehouse
. * JDBC Database Name: In StarRocks 3.x, try specifying the full path in your connection string if the app is losing catalog context:
jdbc:mysql://<host>:<port>/default_catalog.<db_name>
. 5. Application User Resource Groups If you have Resource Isolation configured, the application user might be mapped to a Resource Group that is restricted from using the
default_warehouse
or has zero allocated resources. * Check:
SHOW RESOURCE GROUPS;
and see if there are any classifiers matching your application's
user
or
host
. Recommended Debug Step: Run the following inside your application (e.g., via a log statement or a temporary debug endpoint) and compare the output to your GUI:
Copy code
sql
SELECT current_user();
SELECT current_role();
SHOW GRANTS;
SHOW COMPUTE NODES; -- See if 'Alive' is true from the App's perspective
If
SHOW COMPUTE NODES
returns an empty list or
Alive: false
when run from the application but works from the GUI, you have a network/routing issue between the specific FE the app is hitting and the compute layer. Referencessql-reference/sql-functions/utility-functions/current_warehouse.mdsql-reference/information_schema/warehouse_queries.md
k
@Rocky Id Name State NodeCount CurrentClusterCount MaxClusterCount StartedClusters RunningSql QueuedSql CreatedOn ResumedOn UpdatedOn Property Comment 0 default_warehouse AVAILABLE 0 1 1 1 0 0 An internal warehouse init after FE is ready
r
Hello, @Kenneth Oh! How can I assist you today?
The output you provided is the key:
NodeCount
is 0
. Even though the warehouse state is
AVAILABLE
, it has no compute power. In StarRocks v3.x (Shared-Data), a warehouse is a logical group of compute nodes (CNs). If
NodeCount
is 0, there are no workers assigned to handle your query.
SELECT current_warehouse()
works because it is a simple metadata check on the Frontend (FE). However, a
SELECT
on a table requires actual hardware (CNs) to scan data, and since your warehouse is empty, it fails. How to Fix This 1. Check if Compute Nodes (CNs) are running Run this command to see if StarRocks sees any compute workers at all:
Copy code
sql
SHOW COMPUTE NODES;
* If the list is empty: Your CN processes are either not started or cannot reach the FE. Check your CN
conf/cn.conf
and ensure the
sys_log_level
is set to INFO to check for connection errors in
log/cn.INFO
. * If nodes are listed but
Alive
is `false`:
The nodes have crashed or the network/firewall is blocking the heartbeat port (default
9050
). * If nodes are listed as
Alive
but
WarehouseName
is not `default_warehouse`:
You have nodes, but they are assigned to a different warehouse. 2. Verify CN Configuration On your compute node machines, check
conf/cn.conf
. By default, CNs try to join
default_warehouse
. If you have a line like this, ensure it matches:
Copy code
properties
# Default is default_warehouse if not specified
warehouse = default_warehouse
3. Why it might work in your GUI but not the App * Metadata vs. Data: Your GUI might only be showing you the list of tables/columns (which comes from the FE metadata and doesn't need CNs). As soon as you try to "Preview Data" or run a
SELECT
, it will likely fail in the GUI too. * User Sessions: If your GUI is logged in as
root
, it might be defaulting to a different warehouse that does have nodes, while your application user is restricted to the empty
default_warehouse
. Check
SELECT current_warehouse()
in both and compare. Summary The
default_warehouse
has 0 nodes. You need to start your CN nodes and ensure they successfully register with the FE. Once
NodeCount
is at least 1, your application queries will work. Useful Debug Commands:
Copy code
sql
-- Check node details and their assigned warehouse
SHOW COMPUTE NODES; 

-- If you see nodes but they aren't in default_warehouse, 
-- you may need to add them (depending on your deployment method)
ALTER WAREHOUSE default_warehouse ADD "cn_ip:heartbeat_port";
Referencessql-reference/sql-statements/cluster-management/nodes_processes/SHOW_COMPUTE_NODES.mdsql-reference/sql-functions/utility-functions/current_warehouse.mdadministration/user_privs/authorization/privilege_faq.mdadministration/user_privs/authorization/User_privilege.md Referencestable_design/table_design.mdquick_start/iceberg.mddata_source/catalog/default_catalog.mdquick_start/shared-data.mdadministration/Meta_recovery.mdknowledge/admin_and_operation/shared_data_compaction.md
k
@Rocky how do i make materialized views from a database to not be candidates for rewrite for a user
r
To prevent materialized views (MVs) from a specific database from being candidates for query rewrite for a specific user, you should use StarRocks' Role-Based Access Control (RBAC). In StarRocks, the
SELECT
privilege on a materialized view is specifically required for that MV to be used to "accelerate queries" (i.e., transparent rewrite). If a user does not have
SELECT
privileges on an MV, the optimizer will not consider it a candidate for rewriting their queries. 1. Exclude MVs in a Specific Database (Recommended) To stop all materialized views in
your_db
from being used for rewrite by
your_user
, revoke the
SELECT
privilege on all MVs in that database:
Copy code
sql
-- Ensure you are in the correct catalog context
SET CATALOG default_catalog;

-- Revoke SELECT on all MVs in the specific database
REVOKE SELECT ON ALL MATERIALIZED VIEWS IN DATABASE your_db FROM USER 'your_user'@'%';
After running this, when
your_user
runs a query that matches an MV in
your_db
, the query will execute against the base tables instead of using the MV. 2. Disable Rewrite Globally for a Specific User If you want to disable materialized view rewrite for a specific user across all databases, you can set the
enable_materialized_view_rewrite
session variable as a persistent user property (available from v3.3.3+):
Copy code
sql
-- Disable MV rewrite for this user permanently
ALTER USER 'your_user'@'%' SET PROPERTIES ('session.enable_materialized_view_rewrite' = 'false');
3. Verification To verify that the MVs from the specific database are no longer being used, you can ask the user to run an
EXPLAIN
on their query:
Copy code
sql
EXPLAIN SELECT ... FROM base_table;
* If excluded: The plan will show a scan of the
base_table
. * If still candidate: The plan will show a scan of the
materialized_view_name
. Summary of Privileges Privilege | Effect on Query Rewrite SELECT (on Base Table) | Required to run the query at all. SELECT (on MV) | Required for the optimizer to use the MV for query rewrite. Note: If the user has the
ADMIN
role or has been granted
SELECT
on
ALL DATABASES
, you must revoke those broad privileges or use a more restrictive role for that user to ensure the exclusion takes effect. Referencesadministration/user_privs/authorization/privilege_item.mdsql-reference/sql-statements/account-management/REVOKE.mdsql-reference/sql-statements/account-management/GRANT.mdsql-reference/sql-statements/materialized_view/DROP_MATERIALIZED_VIEW.mdsql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md
k
@Rocky how to selectively revoke select mvs on a database
r
To selectively revoke the
SELECT
privilege on materialized views (MVs) in a specific database, you can use the following commands depending on your needs. 1. Revoke SELECT on ALL Materialized Views in a Database If you want to stop a user from using any materialized view in a specific database for query rewrite or direct querying:
Copy code
sql
-- You must be in the correct catalog context
SET CATALOG default_catalog;

REVOKE SELECT ON ALL MATERIALIZED VIEWS IN DATABASE <db_name> FROM USER 'user_identity';
2. Revoke SELECT on Specific Individual Materialized Views If you want to revoke privileges only for specific MVs while allowing others in the same database to remain active for rewrite:
Copy code
sql
-- Revoke from specific MVs by listing them
REVOKE SELECT ON MATERIALIZED VIEW <mv_name1>, <mv_name2> IN DATABASE <db_name> FROM USER 'user_identity';

-- Or use the fully qualified name
REVOKE SELECT ON MATERIALIZED VIEW <db_name>.<mv_name> FROM USER 'user_identity';
Important Considerations: * Query Rewrite: In StarRocks, the
SELECT
privilege on an MV is the specific permission required for the optimizer to use that MV for transparent query rewrite. Revoking
SELECT
is the standard way to "hide" an MV from a specific user's query optimizer. * Catalog Context: Always ensure you have set the correct catalog (e.g.,
SET CATALOG default_catalog;
) before running these commands, as MVs are catalog-specific objects. * User Identity: The
'user_identity'
should be in the format
'username'@'host'
(e.g.,
'report_user'@'%'
). How to Verify To confirm the privileges have been removed, you can check the grants for that user:
Copy code
sql
SHOW GRANTS FOR 'user_identity';
Referencessql-reference/sql-statements/account-management/REVOKE.mdsql-reference/sql-statements/account-management/GRANT.mdadministration/user_privs/authorization/privilege_item.md Referencesadministration/user_privs/authorization/privilege_item.mdsql-reference/sql-statements/account-management/GRANT.mdsql-reference/sql-statements/account-management/REVOKE.mdsql-reference/sql-statements/materialized_view/DROP_MATERIALIZED_VIEW.mdadministration/user_privs/authorization/User_privilege.md
k
@Rocky GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE default_catalog.starrocks TO USER 'user'; results in: Query 1 ERROR at Line 9: : Getting syntax error at line 1, column 66. Detail message: Unexpected input '.', the most similar input is {'TO'}.
r
The syntax error occurs because the
GRANT
statement on database-level objects (like
ALL MATERIALIZED VIEWS
) does not support the
catalog.database
dot notation. In StarRocks, you must first switch to the target catalog using
SET CATALOG
and then run the grant command using only the database name. How to Fix Run the commands in two steps:
Copy code
sql
-- 1. Switch to the catalog
SET CATALOG default_catalog;

-- 2. Run the GRANT command using only the database name
GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE starrocks TO USER 'user'@'%';
Key Rules * Catalog Context: For any
GRANT
command involving
IN DATABASE <db_name>
, you must run
SET CATALOG
first. * User Identity: Ensure you use the full user identity (e.g.,
'user'@'%'
) as it was defined during
CREATE USER
. * Dot Notation: While you can use
db.mv_name
for a single materialized view, the
ALL ... IN DATABASE
syntax strictly expects a simple database name. Referencessql-reference/sql-statements/account-management/GRANT.mdadministration/user_privs/authorization/privilege_item.md Referencessql-reference/sql-statements/account-management/GRANT.mdadministration/user_privs/authorization/privilege_item.md
k
@Rocky is there a way to monitor nvme effectiveness? would r8gd.4xlarge vs r8g.4xlarge perform drastically different in a shared data cluster with ample ebs disk for cache?
r
To monitor the effectiveness of your NVMe cache and compare the performance of
r8gd
vs.
r8g
instances in a StarRocks shared-data cluster, you should focus on specific I/O metrics and understanding the architectural trade-offs between local instance storage and network-attached EBS. 1. How to Monitor NVMe Cache Effectiveness StarRocks provides multiple layers of observability for the Data Cache (which uses your local disks to cache data from S3/remote storage). Query Profile (Fine-grained) For any specific query, check the Query Profile (using
EXPLAIN ANALYZE
). Look for these metrics in the Scan nodes: *
DataCacheReadBytes
: Data read directly from the local disk/NVMe cache. *
DataCacheWriteBytes
: Data fetched from remote storage (S3) and written to the cache. *
DataCacheReadTimer
: The latency of reading from your local disk. If this is high (e.g., >10ms), your disk is a bottleneck. Backend Metrics (Global) You can access real-time statistics via the BE HTTP API:
Copy code
bash
curl http://<BE_IP>:<BE_HTTP_PORT>/api/datacache/stat
Key fields to watch: *
block_cache_hit_rate
: The percentage of data requests satisfied by the disk cache. *
block_cache_read_disk_bytes
: Total throughput from the NVMe/EBS disks. *
block_cache_write_fail_count
: If this is non-zero, your disks may be overloaded or full. Prometheus/Grafana If using the StarRocks Grafana dashboard, monitor: *
fslib cache hit ratio
* `fslib read io_latency`: Crucial for comparing NVMe vs. EBS. NVMe should typically be sub-millisecond, while EBS (GP3) often sees 1–5ms+ latencies. ────────── 2.
r8gd.4xlarge
vs. `r8g.4xlarge`: Is there a drastic difference?
Yes, there is often a drastic performance difference in a shared-data cluster when your working set exceeds your RAM capacity. Feature |
r8g.4xlarge
(EBS Only)
|
r8gd.4xlarge
(Local NVMe)
Latency | Network-attached (1ms - 10ms+) | Physically attached (<100μs - 500μs) Throughput | Capped by EBS/Instance bandwidth (e.g., 10Gbps) | High local bus speed (Multiple GB/s) Cost | Pay for EBS (GB/IOPS/Throughput) | Included in instance price When they perform the same: If your working set is small enough to fit entirely in the Page Cache (RAM), the disk performance won't matter much. Both instances have the same CPU (Graviton4) and RAM (128GB). When
r8gd
(NVMe) wins "drastically":
1. Massive Scans: In OLAP workloads, StarRocks frequently performs "small-block random reads" (especially for metadata and specific column segments). EBS suffers from "network jitter" and higher base latency for these small IOPS. NVMe handles these almost instantly. 2. High Concurrency: When many queries run simultaneously, EBS throughput and IOPS limits are easily hit, leading to "I/O Wait" and query queuing. Local NVMe has significantly higher ceilings. 3. Cache Warmup/Filling: When first loading data from S3 to the cache, the
r8gd
can write to the local NVMe much faster than an
r8g
can write to EBS without competing with other network traffic. Recommendation For a shared-data cluster,
r8gd
is almost always the superior choice
. In a shared-data architecture, the local disk is essentially a "disposable" performance accelerator. Since
r8gd
includes high-performance NVMe at a slight price premium over
r8g
, the gain in query stability and sub-second response times usually outweighs the cost, especially compared to the expense of over-provisioning EBS GP3/IO2 to match NVMe performance. Referencesusing_starrocks/caching/block_cache.mddeployment/shared_data/feature-support-shared-data.mddata_source/data_cache_observe.md
k
@Rocky when doing cache select, is cache loaded into nvme or ebs
r
When you perform a
CACHE SELECT
(or any query that triggers caching) in a shared-data cluster, the data is loaded into the storage location specified by the
storage_root_path
configuration in your
cn.conf
or
be.conf
. Whether that data ends up on NVMe or EBS depends entirely on which disk is mounted to that path on your OS. How it works: 1. Configuration determines the path: StarRocks looks at the
storage_root_path
(default is
${STARROCKS_HOME}/storage
). 2. Infrastructure determines the hardware: * If you mount your Local NVMe (on an
r8gd
instance) to
/data/cache
and set
storage_root_path = /data/cache
, the cache is loaded into NVMe. * If you mount an EBS volume to that path, it is loaded into EBS. 3. Data Placement: The actual cache files are stored in a sub-directory named
starlet_cache/
under your root path. Key Recommendation for r8gd vs r8g: * On
r8gd
(with NVMe):
You should always point
storage_root_path
to the mount point of your local NVMe. This provides the lowest latency for the Data Cache, which is the primary reason to use
d
-series instances. * On
r8g
(EBS only):
Since there is no local NVMe, you must point it to an EBS volume. Because EBS is network-attached, your "cache hits" will be limited by EBS bandwidth and network latency. How to verify where your cache is going: You can check which directory StarRocks is using for the cache by running:
Copy code
sql
SELECT * FROM information_schema.be_configs WHERE NAME = 'storage_root_path';
Then, on your Linux terminal, run
df -h <path>
for that directory to see which device (NVMe or EBS) it is mapped to. Referencesdeployment/shared_data/s3.mdusing_starrocks/caching/block_cache.mddeployment/shared_data/feature-support-shared-data.mdsql-reference/sql-statements/materialized_view/CACHE_SELECT.md Referencesdeployment/shared_data/s3.mdusing_starrocks/caching/block_cache.mddeployment/shared_data/feature-support-shared-data.mddeployment/shared_data/hdfs.mddeployment/shared_data/gcs.mddeployment/shared_data/minio.mddeployment/shared_data/azure.mdsql-reference/sql-statements/materialized_view/CACHE_SELECT.mdadministration/management/BE_configuration.mdsql-reference/sql-statements/data-definition/CREATE_TABLE.md Referencesusing_starrocks/caching/block_cache.mddeployment/shared_data/s3.mddeployment/shared_data/hdfs.mddeployment/shared_data/gcs.mddeployment/shared_data/minio.mddeployment/shared_data/feature-support-shared-data.md
k
@Rocky does this mean cache is giong into nvme: 1746451 storage_root_path /mnt/nvme/storage;/mnt/ebs/storage string ${STARROCKS_HOME}/storage 0 and what happens when nvme is filled up?
r
Yes, with that configuration, your cache is going into both the NVMe and the EBS disks. 1. How Multiple Paths Work When you list multiple directories in
storage_root_path
separated by a semicolon, StarRocks treats them as a storage pool for the Data Cache. * Distribution: StarRocks will create a
starlet_cache/
(or
datacache/
) directory on both
/mnt/nvme/storage
and
/mnt/ebs/storage
. * Performance Warning: Because EBS is significantly slower than NVMe, your query performance may become inconsistent. Some data blocks will be served at sub-millisecond speeds from NVMe, while others will take several milliseconds from EBS. It is generally recommended to use only the NVMe disk for the cache on
r8gd
instances to ensure peak, consistent performance. 2. What happens when the NVMe fills up? StarRocks has a built-in self-management mechanism for the Data Cache: * LRU Eviction: The cache does not wait for the disk to be 100% full. By default, it uses a quota (controlled by
starlet_star_cache_disk_size_percent
, which defaults to 80%). * Overwriting: When the cache reaches that 80% threshold, StarRocks uses a Least Recently Used (LRU) policy. It doesn't "delete" files from the disk (so
df -h
will still show the disk as "full"), but it marks the oldest blocks as overwritable. New data coming in from S3 will simply overwrite the oldest cached data. * Disk Selection: If you have multiple disks, StarRocks will continue to write to whichever disk has available cache quota. If the NVMe portion of the cache is "full" (at 80%), it will trigger eviction on that disk. Recommendation If your local NVMe is large enough to hold your "hot" data, you should remove the EBS path from `storage_root_path`:
Copy code
properties
-- Optimized for r8gd
storage_root_path = /mnt/nvme/storage
This ensures that 100% of your cached data benefits from NVMe speeds. Only include the EBS path if your hot data set is much larger than the NVMe capacity and you are willing to accept the slower EBS latency for the overflow. Referencesusing_starrocks/caching/block_cache.mdadministration/management/BE_configuration.mddeployment/shared_data/s3.mddata_source/data_cache_troubleshooting.mdfaq/Deploy_faq.mddeployment/shared_data/feature-support-shared-data.mddeployment/shared_data/minio.mddeployment/shared_data/gcs.mddeployment/shared_data/azure.mddeployment/shared_data/hdfs.mdsql-reference/sql-statements/materialized_view/CACHE_SELECT.mdfaq/operation_maintenance_faq.mdusing_starrocks/caching/query_cache.mddata_source/data_cache.md Referencesusing_starrocks/caching/block_cache.mdadministration/management/BE_configuration.mddeployment/shared_data/s3.md