<@U0A71G31CDV> hey rocky,i have this error * Abor...
# questions-and-troubleshooting
f
@Rocky hey rocky,i have this error * Aborted at 1780813912 (unix time) try "date -d @1780813912" if you are using GNU date * PC: @ 0x10904feb bshuf_default_block_size_avx2 * SIGFPE (@0x10904feb) received by PID 28 (TID 0x7f5978df6640) LWP(1090) from PID 277893099; stack trace: * @ 0x7f5b3cd48ee8 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x99ee7) @ 0x10105149 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) @ 0x7f5b3ccf1520 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x4251f) @ 0x10904feb bshuf_default_block_size_avx2 @ 0x109051b0 bshuf_blocked_wrap_fun_avx2 @ 0x10905607 bshuf_decompress_lz4_avx2 @ 0x9793464 starrocks:BitShuffleDataDecoder:decode_page_data(starrocks::PageFooterPB*, unsigned int, starrocks::EncodingTypePB, std::unique_ptr<char [], std::default_delete<char []> >*, starrocks::Slice*) @ 0x9791d55 starrocks:StoragePageDecoder:decode_page(starrocks::PageFooterPB*, unsigned int, starrocks::EncodingTypePB, std::unique_ptr<char [], std::default_delete<char []> >*, starrocks::Slice*) @ 0x978d5e4 starrocks::read_and_decompress_page_internal(starrocks::PageReadOptions const&, starrocks::PageHandle*, starrocks::Slice*, starrocks::PageFooterPB*) @ 0x978ddd3 starrocks:PageIO:read_and_decompress_page(starrocks::PageReadOptions const&, starrocks::PageHandle*, starrocks::Slice*, starrocks::PageFooterPB*) @ 0x9774b2f starrocks:ColumnReader:read_page(starrocks::ColumnIteratorOptions const&, starrocks::PagePointer const&, starrocks::PageHandle*, starrocks::Slice*, starrocks::PageFooterPB*) @ 0x983cb9b starrocks:ScalarColumnIterator:_read_data_page(starrocks::OrdinalPageIndexIterator const&) @ 0x983ecc6 starrocks:ScalarColumnIterator:_load_next_page(bool*) @ 0x983f45c starrocks:ScalarColumnIterator:next_batch(starrocks::SparseRange<unsigned int> const&, starrocks::Column*) @ 0x9c430cb starrocks:SegmentIterator:_read(starrocks::Chunk*, std::vector<unsigned int, std::allocator<unsigned int> >*, unsigned long) @ 0x9c0d21f starrocks:SegmentIterator:_do_get_next(starrocks::Chunk*, std::vector<unsigned int, std::allocator<unsigned int> >*) @ 0x9c1c682 starrocks:SegmentIterator:do_get_next(starrocks::Chunk*, std::vector<unsigned long, std::allocator<unsigned long> >*) @ 0x9c806f8 starrocks:SegmentIterator:do_get_next(starrocks::Chunk*, std:vector&lt;starrocks:RowSourceMask, std:allocator&lt;starrocks:RowSourceMask> >*, std::vector<unsigned long, std::allocator<unsigned long> >*) @ 0xa7a5fb7 starrocks:MergeEntry&lt;starrocksSlice&gt;:next() @ 0xa7a8f30 starrocks:RowsetMergerImpl&lt;starrocksSlice&gt;:_fill_heap(starrocks:MergeEntry&lt;starrocks:Slice>*) @ 0xa7a963e starrocks:RowsetMergerImpl&lt;starrocksSlice&gt;:get_next(starrocks::Chunk*, std:vector&lt;starrocks:RowSourceMask, std:allocator&lt;starrocks:RowSourceMask> >*, std::vector<unsigned long, std::allocator<unsigned long> >*) @ 0xa7aa79b starrocks:RowsetMergerImpl&lt;starrocksSlice&gt;:_do_merge_horizontally(starrocks::Tablet&, std:shared ptr&lt;starrocks:TabletSchema const> const&, long, starrocks::Schema const&, std:vector&lt;stdshared ptr&lt;starrocks:Rowset>, std:allocator&lt;std:shared_ptr<Q @ 0xa7ad731 starrocks:RowsetMergerImpl&lt;starrocksSlice&gt;:do_merge(starrocks::Tablet&, std:shared ptr&lt;starrocks:TabletSchema const> const&, long, starrocks::Schema const&, std:vector&lt;stdshared ptr&lt;starrocks:Rowset>, std:allocator&lt;stdshared ptr&lt;starrocks:RowQ @ 0xa79f244 starrocks::compaction_merge_rowsets(starrocks::Tablet&, long, std:vector&lt;stdshared ptr&lt;starrocks:Rowset>, std:allocator&lt;stdshared ptr&lt;starrocks:Rowset> > > const&, starrocks::RowsetWriter*, starrocks::MergeConfig const&, std:shared ptr&lt;starrocks:Q @ 0x96fe7a5 starrocks:TabletUpdates:_do_compaction(std:unique ptr&lt;starrocks:CompactionInfo, std:default delete&lt;starrocks:CompactionInfo> >*, std::
r
Based on the stack trace you provided, the StarRocks Backend (BE) is crashing with a
SIGFPE
(Floating Point Exception) inside the BitShuffle decompression logic (
bshuf_default_block_size_avx2
) during compaction (
TabletUpdates::_do_compaction
). Root Cause Analysis A
SIGFPE
at this specific location typically indicates a division-by-zero error within the BitShuffle library. This happens when the library attempts to calculate the processing block size by dividing a target buffer size by the element size of the column data. If the element size passed to the decoder is 0, the process crashes. This can occur due to: 1. Metadata Corruption: The segment file's footer or the column metadata on disk has been corrupted, leading the BE to read an element size of 0. 2. Schema/Type Inconsistency: A bug in a specific StarRocks version where certain data types (often complex types like
ARRAY
or variable-length types) are handled incorrectly during the compaction merge phase. 3. Instruction Set Issues: While less likely for
SIGFPE
, if the environment (e.g., Docker/VM) incorrectly reports AVX2 support or has an incompatible CPU, SIMD-optimized math libraries can sometimes behave unpredictably. Recommended Steps 1. Identify the problematic Tablet Check the
<http://be.INFO|be.INFO>
or
be.out
log file immediately preceding the crash. Look for a line similar to:
[tablet_updates.cpp:...] start to do compaction for tablet: <TABLET_ID>
Identifying the
TABLET_ID
allows you to pinpoint which table and data partition are causing the crash. 2. Check for Metadata Inconsistency Once you have the
TABLET_ID
, run the following in your MySQL client:
Copy code
sql
ADMIN SHOW REPLICA STATUS FROM <table_name> WHERE TabletId = <TABLET_ID>;
-- Verify if the replica is healthy and check its version.
You can also try to force a check on the tablet:
Copy code
sql
ADMIN CHECK TABLET (<TABLET_ID>);
3. Workaround: Disable BitShuffle (If applicable) If this is a recurring issue on a specific table and you can recreate it, try using a different encoding (like
PLAIN
or
LZ4
without BitShuffle) for numeric columns. BitShuffle is often the default for integers/floats. 4. Version & Environment * Upgrade: If you are on an older version (e.g., 2.x or early 3.x), this might be a known bug in the storage engine's metadata handling. Check if there is a newer patch release. * CPU Compatibility: Ensure your BE nodes are running on CPUs that fully support AVX2. If you are mixing different CPU architectures in one cluster, this is not recommended. Next steps: If the crash persists after identifying and repairing the problematic tablet, please provide your StarRocks version (
SELECT current_version();
) and the table schema to help narrow down if this is a specific data-type bug. Referencesfaq/Deploy_faq.mdfaq/Others.mddevelopers/build-starrocks/Build_in_docker.mdsql-reference/sql-functions/hash-functions/xx_hash3_64.mdsql-reference/sql-functions/bit-functions/bit_shift_right_logical.mdsql-reference/sql-functions/bitmap-functions/bitmap_hash64.mdsql-reference/sql-functions/bit-functions/bit_shift_right.mdadministration/Meta_recovery.md