http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> On a single-node shared-nothing Sta...
# questions-and-troubleshooting
m
@Rocky On a single-node shared-nothing StarRocks setup (Docker), a column-list INSERT into a Primary Key table hangs indefinitely. The FE commits the transaction, but the BE fails to apply it and the tablet goes into an error state. Version: 
4.11
Table: sql
Copy code
CREATE TABLE test_database.`MetaDataService_Control_Bineet4` (
  `property` varchar(255) NOT NULL COMMENT "",
  `intValue` int(11) NOT NULL DEFAULT "0" COMMENT "",
  `stringValue` varchar(255) NULL COMMENT ""
) ENGINE=OLAP
PRIMARY KEY(`property`)
DISTRIBUTED BY HASH(`property`) BUCKETS 1
PROPERTIES (
"compression" = "LZ4",
"enable_persistent_index" = "true",
"fast_schema_evolution" = "true",
"replicated_storage" = "false",
"replication_num" = "1"
);
Statement that hangs: sql
Copy code
INSERT INTO test_database.MetaDataService_Control_Bineet4 (`property`, `intValue`) VALUES ('updateCount', 0);
FE log: the transaction commits successfully:
Copy code
begin transaction: txn_id: 2 with label insert_01a0ec18-67ef-736a-95fb-6f5c8788c30e
transaction:[TransactionState. txn_id: 2, ... db id: 10001, table id list: 10277, ... transaction status: COMMITTED ... partition commit info:[partitionId=10279, version=2 ...]] successfully committed
BE log: apply fails right after the commit:
Copy code
I tablet_updates.cpp:694] commit rowset tablet:10280 version:2 txn_id:2
I task_worker_pool.cpp:612] publish version finished transactions: [2,]
E tablet_updates.cpp:1395] _apply_rowset_commit error: apply rowset update state failed: Corruption: Bad segment file /opt/starrocks/be/storage/data/110/10280/1053825165/020000000000001b1e46ac16db859c2d9bf1740d07297abe_0.dat: magic number not match
be/src/storage/rowset/segment_rewriter.cpp:47 Segment::parse_segment_footer(rfile.get(), &footer, nullptr, &partial_rowset_footer)
be/src/storage/rowset_update_state.cpp:800 SegmentRewriter::rewrite_partial_update(src, &dest, _tablet_schema, read_column_ids, _partial_update_states[segment_id].write_columns, segment_id, partial_rowset_footer) tablet:10280 #version:2 [1 1@0 2] pending: rowsets:1
  0 [seg:1 row:1 del:0 bytes:383 row_size:19 compaction_score:268435073 compaction_level:-1 partial_update_by_column:false]
E tablet_updates.cpp:1063] apply tablet: 10280 failed, status: Corruption: _apply_rowset_commit error: ...
W tablet_updates.cpp:2751] remove_expired_versions failed, tablet updates is in error state: tablet:10280 ...
Questions: 1. Is this a known bug in the partial-update path ? 2. Is there an FE/BE config or session setting that avoids the partial-update path for column-list INSERTs, without changing the SQL?
r
The behavior you are seeing is a result of how StarRocks (v3.x and v4.x) handles 
INSERT
statements that specify only a subset of columns for a Primary Key table. 1. Is this a known bug? The error 
Corruption: Bad segment file ... magic number not match
in 
rewrite_partial_update
indicates that the BE is failing to parse the footer of a segment file during the Partial Update merge process. This is a known class of issues in the Primary Key "Partial Update" path (introduced to support 
UPSERT
semantics via 
INSERT
). Specifically, when an 
INSERT
does not include all columns, StarRocks triggers a "read-on-write" or "rewrite" operation to merge the new data with existing rows (or default values). In version 4.x, improvements to the layout and segment splitting (mentioned in recent 4.1.x release notes) have occasionally triggered edge cases in the segment footer parsing during these rewrites. 2. How to avoid the partial-update path You can bypass the partial-update logic (the "rewrite" path) by changing a system variable. This will cause StarRocks to treat the missing columns as 
NULL
or their 
DEFAULT
values immediately, rather than trying to preserve existing values in the table. Session/Global Config: Set the following variable to 
false
before running your `INSERT`:
Copy code
sql
SET enable_insert_partial_update = false;
* When 
true
(Default): 
INSERT INTO pk_table (col_a) VALUES (1)
acts as a partial update. It looks up existing rows and preserves values for 
col_b
, 
col_c
, etc. * When `false`: The statement acts as a standard 
UPSERT
. It inserts 
1
for 
col_a
and the default/null values for all other columns, overwriting the entire row if the primary key exists. This avoids the 
rewrite_partial_update
code path entirely. Alternative (Statement-level hint): If you cannot change the session, you can use a hint (supported in v3.4.0+ and 4.x):
Copy code
sql
INSERT /*+ SET_VAR(enable_insert_partial_update = false) */ 
INTO test_database.MetaDataService_Control_Bineet4 (`property`, `intValue`) 
VALUES ('updateCount', 0);
Recommendation 1. Immediate Fix: Set 
enable_insert_partial_update = false
. This should stop the "magic number" corruption error and the hanging, as it bypasses the buggy rewrite logic. 2. Health Check: Since the BE log shows 
tablet:10280 updates is in error state
, you should check the health of that tablet. Use 
SHOW TABLET 10280;
and check if it needs manual repair or if the BE restart clears the error state. 3. Upgrade: Given you are on a very early version of the 4.x line (e.g., 
4.1.1
), check for a patch release like 
4.1.5
or later, which includes multiple fixes for "lake column updates" and "segment rewrite" stability. References • using_starrocks/async_mv/troubleshooting_asynchronous_materialized_views.md • sql-reference/sql-statements/table_bucket_part_index/UPDATE.md • faq/Others.md • using_starrocks/async_mv/use_cases/query_rewrite_with_materialized_views.md • release_notes/release-2.4.md
Open in Slack
PreviousNext
Powered by