http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> can stats collection operations fai...
# questions-and-troubleshooting
j
@Rocky can stats collection operations fail due to recent or past schema migrations? Seeing a "Statistics query fail | Error Message [[reason=INTERNAL_ERROR] [msg=column type mismatch: BE:12712988]] |" for a statistics collection, happened intermittently, stopped happening
r
Yes, statistics collection operations can fail intermittently due to recent or past schema migrations. The error message you encountered—`[msg=column type mismatch: BE:...]`—typically indicates a transient inconsistency between the Frontend (FE) metadata and the Backend (BE) physical data layout during or immediately after a schema change. Why this happens 1. Metadata vs. Data Synchronization: When you perform a schema change (especially a Light Schema Change in StarRocks), the FE updates its metadata immediately. However, the physical segments on the BEs might still be transitioning or have different versioned schemas. If a statistics collection task (which runs a background scan) is triggered during this window, the FE may request a column type that the BE's current version of the data does not yet match. 2. Intermittent Nature: This is common because statistics collection often uses sampled scanning or hits specific replicas. If only some BEs or some tablets have completed the schema transition, the error will appear intermittently depending on which replica the statistics job attempts to scan. 3. Light Schema Change (LSC): StarRocks uses LSC for operations like adding or dropping value columns. In these cases, schema consistency is managed during reads and writes rather than through a heavy data rewrite. A race condition or a delay in BE schema cache refreshing can lead to a temporary "type mismatch" error. Why it "stopped happening" The error usually resolves itself once: * The schema change job fully completes across all replicas. * The BEs refresh their internal schema caches. * The FE and BE reach a consistent state regarding the column definitions. Troubleshooting and Verification If you see this again, you can check the status of your schema migrations and table health: * Check Schema Change Status: Run 
SHOW ALTER TABLE COLUMN;
to see if any migration jobs are still in 
RUNNING
or 
FINISHED
states from that timeframe. * Check Table Health: Use 
SHOW TABLE STATUS;
or check the 
_statistics_.column_statistics
table to see if any columns have stale or failed stats. * Manual Trigger: If statistics for that table are missing, you can manually re-trigger collection to ensure everything is now consistent:
Copy code
sql
    ANALYZE TABLE <table_name>;
Summary: Since the error stopped occurring, it was almost certainly a transient state caused by a schema migration. If the table is performing normally and 
ANALYZE
jobs are now succeeding (check via 
SHOW ANALYZE JOB
), no further action is action is no further action is required. References • administration/management/monitoring/metrics.md • administration/management/monitoring/Monitor_and_Alert.md • using_starrocks/Cost_based_optimizer.md
Yes, statistics collection operations can fail intermittently due to recent or past schema migrations. The error message you encountered—`[msg=column type mismatch: BE:...]`—typically indicates a transient inconsistency between the Frontend (FE) metadata and the Backend (BE) physical data layout during or immediately after a schema change. Why this happens 1. Statistics Collection is a Query: StarRocks collects statistics by running internal background queries (e.g., 
SELECT COUNT(*), MIN(col)...
). Like any other query, these tasks rely on the FE and BE agreeing on the table's schema. 2. Metadata vs. Data Synchronization: When you perform a schema change (such as 
ALTER TABLE
), the FE updates its metadata immediately. However, the physical segments on the BEs may take time to update their local tablet schema caches or finish versioning. If a statistics task is triggered during this window, the FE may request a column type (the "new" type) that the BE still sees as the "old" type, triggering the 
column type mismatch
. 3. Light Schema Change (LSC): StarRocks often uses LSC to avoid full data rewrites. During this process, different versions of the data may exist across different replicas or rowsets. If the statistics job—which often uses *sampled scanning*—hits a replica or tablet that hasn't fully synchronized the schema update, the error occurs. 4. Intermittent Nature: The error is intermittent because it depends on which specific BE or tablet replica the statistics query happens to target. Once all replicas and BE caches are updated, the error disappears. Why it "stopped happening" The error usually resolves itself automatically once: * The schema change job fully completes (status 
FINISHED
in 
SHOW ALTER TABLE COLUMN;
). * The BEs refresh their internal schema caches. * The FE and BE reach a consistent state regarding the column definitions. Recommended Actions Since the error has stopped, the system has likely reached consistency. However, you can verify the following: * Check Schema Change Status: Run 
SHOW ALTER TABLE COLUMN;
to confirm that all recent migrations are finished and didn't leave any tablets in an error state. * Verify Statistics: Check if the statistics were actually collected for that table:
Copy code
sql
    SHOW ANALYZE STATUS WHERE table_name = 'your_table_name';
* Manual Trigger: If you find that statistics are missing or "Stale," you can manually re-trigger a full collection now that the schema is stable:
Copy code
sql
    ANALYZE TABLE <table_name>;
If this happens frequently after every schema change, you may want to check if you are on an older version of StarRocks (pre-3.x), as newer versions have improved the synchronization of schema metadata during background tasks. References • administration/management/monitoring/metrics.md • using_starrocks/Cost_based_optimizer.md • sql-reference/information_schema/statistics.md • knowledge/usage/best-practice.md • loading/minio.md
2 Views
Open in Slack
PreviousNext
Powered by