@Rocky We hit a wrong-result issue on a primary-key table and wanted to check whether it's known and which version fixes it. ***Setup*** - StarRocks 4.1.1 (

4.1.1-14b7e3f

), shared-nothing, 6 BEs, aarch64 - PK table

dim

, ~8M rows:

PRIMARY KEY (k1, k2) DISTRIBUTED BY HASH(k2) BUCKETS 1

,

replication_num = 3

, not partitioned - Globals:

partial_update_mode = column

,

enable_global_runtime_filter = true

,

runtime_join_filter_push_down_limit = 1024000

,

enable_predicate_col_late_materialize = false

,

push_down_heavy_exprs = false

***What we did to the table in one day*** 1. ~1M full-row

INSERT … SELECT

upserts (new keys) plus ~1M

DELETE … USING

2. ~4.7M full-row inserts of new keys 3. ~1.9M

DELETE … USING

4. A column-list

INSERT INTO dim (k1, k2, col_x) SELECT …

partial update of ~1.25M rows ***Symptom (after step 4)*** Joining a fact table's keys to it (

fact LEFT JOIN dim ON k1, k2

) matched ~217.6k of ~254.8k keys, consistently on every run. The same query with runtime filters disabled for the session matched all ~254.7k:

sql
-- misses ~37k matches
WITH k AS (SELECT DISTINCT k1, k2 FROM fact WHERE ...)
SELECT COUNT(*), COUNT(d.k2) FROM k LEFT JOIN dim d
  ON d.k1 = k.k1 AND d.k2 = k.k2;

-- correct
SET enable_global_runtime_filter = false;
SET runtime_join_filter_push_down_limit = 0;
-- same query

A direct point lookup (

WHERE k1 = 'x' AND k2 = 'y'

) returned a row that the join said didn't exist.

COUNT(*)

and

COUNT(IF(col_x IS NULL, 1, NULL))

on the table itself were stable and correct. Before step 4, the same join matched everything. ***Workaround***

INSERT INTO dim SELECT * FROM dim

(a full self-reinsert). Afterwards the join with default settings matches all rows again. ***Questions*** 1. Is this a known issue with runtime filters on PK tables after column-mode partial updates? Is there an issue or PR, and is it fixed in 4.1.4 or 4.1.5? 2. Is

partial_update_mode = column

the likely trigger, or can large upsert/delete volumes alone cause it? 3. Is there a way to detect affected tablets other than comparing join results with runtime filters on and off?

Based on the symptoms and the provided setup (StarRocks 4.1.1, Primary Key table, and

partial_update_mode = column

), this appears to be a known issue related to the Delta Column Group (DCG) mechanism used for column-mode partial updates. 1. Is this a known issue? Is it fixed in 4.1.4/4.1.5? Yes, this is a known category of issues where features relying on column-level metadata or specific scan optimizations (like Runtime Filters and GIN indexes) fail to correctly "see" or process data stored in DCG files after a column-mode partial update. * Related Issues/PRs: * #76271 (Fixed in 4.1.4): Specifically addressed an issue where column-mode partial updates on Primary Key tables returned corrupted results for index-based queries. The fix ensures that data is correctly served from the Delta Column Group segment. * #76744 (Fixed in 4.1.5): Addressed a bug where incorrect ZoneMap pruning produced wrong results. Since Runtime Filters often leverage ZoneMaps or column stats to prune data during the join's build/probe phase, this is a highly likely cause for the missing matches. * #61529: A broader issue tracking cases where DCG data was not read correctly, specifically when switching update modes or during complex scan paths. Recommendation: Upgrade to 4.1.5 or later. Version 4.1.4 and 4.1.5 contain several critical fixes for data visibility and correctness in column-mode partial updates that were present in 4.1.1. 2. Is

partial_update_mode = column

the trigger? Yes. In StarRocks,

column

mode partial updates do not write full new rowsets. Instead, they write "Delta Column Group" (DCG) files containing only the updated columns. * The Primary Key (k1, k2) remains in the original segment, while

col_x

resides in the DCG. * When a Join occurs, if Runtime Filter (RF) pushdown is enabled, the scanner may use metadata (ZoneMaps) to prune data. If the metadata wasn't correctly updated to reflect the "merged" state of the original segment and the DCG, the RF might incorrectly conclude that certain keys do not exist or do not match, causing the join to drop those rows. * Standard row-mode updates (which write full rowsets) or your workaround (a full self-reinsert) bypass this by writing data into standard segments, which explains why the issue disappears after re-inserting. 3. Detection and Workarounds * Detection: Currently, the most reliable way to detect affected tablets is exactly what you did: compare the row counts of joins with and without runtime filters (

SET enable_global_runtime_filter = false

). * Alternative detection: You can check the output of

SHOW TABLE STATUS

or

SHOW TABLET <id>

for tablets with a high number of DCG files, as these are the ones at risk. * Immediate Workarounds: 1. Disable RF for the session:

SET enable_global_runtime_filter = false;

. 2. Force Row Mode: If your update volume allows it, use

partial_update_mode = row

. 3. Trigger Compaction: Manually triggering a cumulative or base compaction on the affected table can help merge DCGs into standard segments, resolving the metadata inconsistency. 4. Self-reinsert: As you discovered,

INSERT INTO dim SELECT * FROM dim

is an effective "nuclear" fix for current data. For production stability, upgrading to 4.1.5 is the suggested path to resolve the underlying DCG metadata bugs. References • sql-reference/System_variable.md • best_practices/primarykey_table.md • table_design/table_types/primary_key_table.md • knowledge/usage/best-practice.md • release_notes/release-1.19.md

link to the image for 4.1.5 please

For StarRocks 4.1.5, you can use the official Docker images from Docker Hub. These images are multi-arch and support the aarch64 (ARM64) architecture used in your setup. Docker Image Tags You can pull the FE and BE images using the following commands: * Frontend (FE):

bash
    docker pull starrocks/fe-ubuntu:4.1.5

* Backend (BE):

bash
    docker pull starrocks/be-ubuntu:4.1.5

Repository Links * Docker Hub: starrocks/fe-ubuntu / starrocks/be-ubuntu * Binary Downloads: If you prefer manual installation, you can find the release binaries on the StarRocks Download Page or directly via the GitHub Releases page. Note on Upgrading: As noted in the 4.1 release documentation, version 4.1 introduces significant changes to data layout. While upgrading from 4.1.1 to 4.1.5 is a standard patch upgrade, ensure you have a metadata backup. After moving to the 4.1 series, downgrading to versions earlier than 4.0.6 is not supported due to metadata compatibility.