@Rocky What is the difference between async population mode and incremental refreshes with materialized views

The terms "Async" and "Incremental" in StarRocks Materialized Views (MVs) refer to two different aspects of how data is kept up to date: the "When/How" (Trigger) and the "What" (Scope). 1. Asynchronous Refresh (The "When/How") This refers to the Asynchronous Materialized View (the modern MV type in StarRocks). Unlike "Synchronous MVs" (Rollups), which are updated in the same transaction as the base table load, Asynchronous MVs decouple the update process. * Behavior: The "population" of the MV happens in the background. When data is loaded into the base table, the MV is not updated immediately. Instead, a background task is triggered later. * Trigger Types: *

REFRESH ASYNC

: Automatically triggered by data changes or on a defined schedule (e.g.,

EVERY(INTERVAL 1 HOUR)

). *

REFRESH MANUAL

: Only triggered when a user runs the

REFRESH MATERIALIZED VIEW

command. * Key Benefit: It prevents data ingestion into base tables from being slowed down by complex MV calculations. 2. Incremental Refresh (The "What/Method") This refers to the Refresh Mode (or granularity), which determines how much data StarRocks processes to bring the MV up to date. * Full Refresh: The system re-calculates the entire MV (or an entire partition) from the base table. * Partitioned Refresh: A form of "incremental" update where only the partitions in the MV that correspond to changed partitions in the base table are refreshed. However, each of those partitions is still fully recomputed. * Incremental Refresh Mode (

"refresh_mode" = "INCREMENTAL"

): This is a specific, optimized mode (introduced in newer versions) where StarRocks performs a row-level or delta-level update. It calculates only the changed rows (inserts/deletes) and applies those specific changes to the MV without re-scanning the entire partition or table. * Key Benefit: Dramatically reduces the resource consumption and time required for updates, especially for large datasets. ────────── Comparison Summary Feature | Asynchronous Refresh | Incremental Refresh Focus | Timing & Trigger. When does the update start? | Scope & Efficiency. How much data is processed? Opposite | Synchronous (Rollup/Sync MV). | Full Refresh (recomputing everything). Typical Property |

REFRESH ASYNC

or

REFRESH MANUAL

|

"refresh_mode" = "INCREMENTAL"

Use Case | Decoupling ingestion from heavy computation. | Minimizing resource usage for large, frequently updated tables. Note on "Async Population" If you are seeing the term "Async Population" specifically, it often refers to: 1.

REFRESH DEFERRED

: When you create an MV, you can specify

DEFERRED

so that the MV is created immediately but its data "population" happens later (asynchronously) via a manual or scheduled refresh. 2. Data Cache: In StarRocks v3.3+, there is a system variable

enable_datacache_async_populate_mode

. This is unrelated to MVs; it controls whether the Block Cache for external tables is filled in the background during a query or synchronously. Recommendation: For large-scale data modeling, use Asynchronous MVs with Partitioned Refresh (using the

PARTITION BY

clause) or the

INCREMENTAL

refresh mode to achieve the best balance of freshness and performance. References • sql-reference/System_variable.md • administration/management/FE_configuration.md • loading/loading_introduction/loading_concepts.md • administration/management/BE_configuration.md • data_source/data_cache.md