http://starrocks.io logo
Join Slack
Powered by
Hi all, I ran into an issue on StarRocks 4.0.6 inv...
# questions-and-troubleshooting
l
Hi all, I ran into an issue on StarRocks 4.0.6 involving a materialized view, and I’d like to ask if anyone has seen something similar or knows whether this is a known bug. Thanks in advance! Materialized view partition erased from recycle bin while still referenced as NORMAL in live metadata → 
tablet does not exist
Environment: StarRocks 4.0.6 (Ubuntu amd64), FE cluster with leader/follower, MV type: ASYNC, UNPARTITIONED. Summary: An async, unpartitioned materialized view started failing all queries with 
failed to get tablet. tablet_id=X, reason=tablet does not exist
, even though 
SHOW PROC '/dbs/.../partitions/...'
on FE reported the replica as 
State=NORMAL
, 
IsBad=false
. The BE’s 
/api/meta/header/{tablet_id}
endpoint confirmed the tablet truly did not exist on disk (
"no tablet exist"
). Timeline (all times GMT+7): • 
2026-09-27 01:46:23
— Manual forced sync refresh (
REFRESH MATERIALIZED VIEW ... FORCE WITH SYNC MODE
) ran successfully, creating a new partition (id 
13131302
) with 10 new tablets (e.g. 
13131362
), loading ~2.42M rows. • 
2026-09-27 01:47:04
/ 
01:58:09
— FE’s 
CatalogRecycleBin.erasePartition()
erased two older partitions of the same table (ids 
13129457
, 
13124718
), consistent with normal ~1-day trash retention from prior refreshes. • 
2026-09-28 02:19:27
— FE’s 
CatalogRecycleBin.erasePartition()
erased partition 
13131302
(the one created at 014623 the previous day) — but this partition was still the live/current partition referenced by the MV’s metadata, not a stale one replaced by a later refresh. • 
2026-09-28 02:21:30
— FE dispatched a batch DROP task for 202 tablets (including the 10 from partition 
13131302
) to the BEs. • 
2026-09-28 02:21:30.19
— BE confirms 
Succeed to drop tablet 13131362
. • 
2026-09-28 03:08
onward — Queries against the MV start failing with 
tablet does not exist
. • Between 
2026-09-28 00:30
and 
03:49
— Seven manual 
REFRESH ... FORCE WITH SYNC MODE
attempts all “succeed” but with 
no partitions to refresh for materialized view
(silent no-op), leaving 
last_refresh_state = SKIPPED
and effectively 0 rows visible, without surfacing any error to the user. • Separately, in the same window, the MV was repeatedly toggled 
inactive
every ~17 min (
mv consecutive failures
) — but this turned out to belong to two other, orphaned materialized view objects with the same name (different table IDs: 
8307289
, 
7743201
) that had each failed hundreds of times and were suspended, apparently left over from earlier MV re-creations. This added confusing/misleading noise while diagnosing the live MV. Core issue: The partition/tablet created by a manual 
REFRESH ... FORCE
was, ~24h later, erased by the recycle bin’s normal expiry cleanup — despite still being the actively referenced partition in the table/MV’s live metadata (not superseded by a newer refresh). This looks like a metadata consistency bug between the live partition state and the recycle bin’s tracked partitions, rather than a disk/BE failure (single BE, 
ReplicationNum=1
, but the tablet was cleanly `DROP`ped by FE’s own request, not lost by the BE). Resolution applied: Dropped and recreated the materialized view; new partition/tablets created successfully, row count matches source, queries succeed again. Questions for the community / StarRocks team: 1. Is there a known scenario where 
CatalogRecycleBin
can erase a partition that is still the “current” partition of a table (not a temp/swap-replaced one)? 2. Could concurrent/overlapping refresh runs (multiple 
FORCE
invocations, or the MV being marked inactive/active repeatedly around the same time) cause the recycle bin to retain a stale reference to a partition ID even after it becomes live again? 3. Is there a relevant fix in 4.0.x releases after 4.0.6 related to 
PartitionBasedMvRefreshProcessor
, 
CatalogRecycleBin
, or temp-partition swap during MV refresh? I can share full 
fe.log
/ 
fe.audit.log
/ 
<http://be.INFO|be.INFO>
excerpts around 
2026-09-27 01:46–02:30
and 
2026-09-28 02:15–02:25
if that helps investigate. Thanks a lot 🙏 @Rocky
@Rocky
r
Hello, @Lê Trọng Quân! How can I assist you today?
The scenario you’ve described indicates a metadata consistency issue where the 
CatalogRecycleBin
incorrectly targeted a live partition for deletion. Based on your timeline and the symptoms (especially the "orphaned" MV objects), this appears to be a known class of metadata collision issues related to naming that were addressed in later patches. 1. CatalogRecycleBin and the "Current" Partition Under normal circumstances, 
CatalogRecycleBin
only erases partitions that have been explicitly dropped (either via 
DROP PARTITION
or by being superseded during an 
INSERT OVERWRITE
/ 
REFRESH MV
swap). However, in versions prior to 4.1, several metadata maps in StarRocks were internally keyed by name/string rather than UUID/Table ID. The presence of your "orphaned" MV objects with the same name is a strong indicator of a metadata clash. If those orphaned objects were suspended or partially cleaned up, the 
CatalogRecycleBin
logic might have incorrectly associated the partition ID of the "live" MV with the deletion task of an "orphaned" one due to a name-based lookup collision. 2. Relevant Fixes and PRs Several fixes in the 4.0.x and 4.1.x branches address these exact symptoms: * PR #76320 (Fixed in v4.1.0): This is the most relevant fix. It changed the internal mapping for materialized view range/partition metadata to be keyed by Table UUID rather than name. This prevents "mishandling of same-named tables" across different databases or after re-creations—exactly what caused your confusing background "consecutive failure" noise. * PR #66501 (Fixed in v4.0.3): Addressed potential issues in 
CatalogRecycleBin
logic for shared-nothing clusters that could cause incorrect metadata erasure. * PR #62634 (Fixed in v3.4.8/v4.0.1): Fixed an issue where residual version mappings from failed refreshes caused subsequent refreshes to be incorrectly 
SKIPPED
, which explains why your subsequent 
FORCE
refreshes resulted in "no partitions to refresh" even though the tablets were missing. 3. Why Refresh "Succeeded" with SKIPPED The "SKIPPED" status and the message 
no partitions to refresh
occur because StarRocks' refresh processor relies on a Version Map. If the tablets are deleted on the BE but the FE metadata still thinks the "version" of the partition is up-to-date with the base table, the refresh logic will skip the task without checking if the physical tablets actually exist. Recommendations * Upgrade to v4.1.x or later: The fix in PR #76320 is crucial for preventing the name-clash that likely triggered the 
CatalogRecycleBin
error. * Manual Cleanup of Orphaned Objects: Before re-creating your MV, ensure no shadow or orphaned objects exist. You can check the 
information_schema.tables
or use 
SHOW PROC '/dbs/<db_id>'
to see if there are multiple entries for the same table name with different IDs. * Adjust Recycle Bin Settings: To provide a safety buffer while you are on 4.0.6, you can increase the retention period for metadata of partitions dropped during refreshes:
Copy code
sql
    ADMIN SET FRONTEND CONFIG ("partition_recycle_retention_period_secs" = "86400"); -- 24 hours
This won't fix the bug, but it will delay the 
CatalogRecycleBin.erasePartition()
call, giving you more time to detect and recover if it happens again. Next Steps for Investigation: If you can still access the 
fe.log
, check for logs from 
CatalogRecycleBin
around 
2026-09-28 02:19
and look for the Table ID it associated with Partition 
13131302
. If that Table ID matches one of your orphaned MVs rather than the live one, it confirms the name-collision bug. References • administration/management/FE_configuration.md • using_starrocks/async_mv/use_cases/create_partitioned_materialized_view.md
Open in Slack
PreviousNext
Powered by