Yoav Nordmann
04/14/2026, 9:06 PMI20260415 02:46:26.790360 140045107308096 daemon.cpp:140] Current memory statistics: process(397565048), query_pool(0), load(0), metadata(17860687), compaction(0), schema_change(0), page_cache(3006937), update
4.0.3 RELEASE (build fbcf73d distro ubuntu arch x86_64)
query_id:019d8d50-f7f9-7676-a9e6-04e2c76cd7d8, fragment_instance:019d8d50-f7f9-7676-a9e6-04e2c76cd7d9, plan_node_id:0
*** Aborted at 1776192387 (unix time) try "date -d @1776192387" if you are using GNU date ***
PC: @ 0x8bc5597 starrocks::BinaryColumnBase<unsigned int>::append(starrocks::Column const&, unsigned long, unsigned long)
*** SIGSEGV (@0x0) received by PID 39 (TID 0x7f5e53251640) LWP(366) from PID 0; stack trace: ***
@ 0x7f5ed47f8ee8 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x99ee7)
@ 0x11d20fc8 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
@ 0x7f5ed47a1520 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x4251f)
@ 0x8bc5597 starrocks::BinaryColumnBase<unsigned int>::append(starrocks::Column const&, unsigned long, unsigned long)
@ 0xc7d3022 starrocks::pipeline::UnionConstSourceOperator::pull_chunk(starrocks::RuntimeState*)
@ 0xae86f7e starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int)
@ 0xc123fee starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0xe1be4de starrocks::ThreadPool::dispatch_thread()
@ 0xe1b5085 starrocks::Thread::supervise_thread(void*)
@ 0x7f5ed47f3ac3 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x94ac2)
@ 0x7f5ed48858c0 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x1268bf)
[1776192387.223][thread: 140043098592832] je_mallctl execute purgeYoav Nordmann
04/14/2026, 9:25 PMRocky
04/14/2026, 9:25 PMYoav Nordmann
04/14/2026, 9:25 PMI20260415 02:46:26.790360 140045107308096 daemon.cpp:140] Current memory statistics: process(397565048), query_pool(0), load(0), metadata(17860687), compaction(0), schema_change(0), page_cache(3006937), update
4.0.3 RELEASE (build fbcf73d distro ubuntu arch x86_64)
query_id:019d8d50-f7f9-7676-a9e6-04e2c76cd7d8, fragment_instance:019d8d50-f7f9-7676-a9e6-04e2c76cd7d9, plan_node_id:0
*** Aborted at 1776192387 (unix time) try "date -d @1776192387" if you are using GNU date ***
PC: @ 0x8bc5597 starrocks::BinaryColumnBase<unsigned int>::append(starrocks::Column const&, unsigned long, unsigned long)
*** SIGSEGV (@0x0) received by PID 39 (TID 0x7f5e53251640) LWP(366) from PID 0; stack trace: ***
@ 0x7f5ed47f8ee8 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x99ee7)
@ 0x11d20fc8 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
@ 0x7f5ed47a1520 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x4251f)
@ 0x8bc5597 starrocks::BinaryColumnBase<unsigned int>::append(starrocks::Column const&, unsigned long, unsigned long)
@ 0xc7d3022 starrocks::pipeline::UnionConstSourceOperator::pull_chunk(starrocks::RuntimeState*)
@ 0xae86f7e starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int)
@ 0xc123fee starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0xe1be4de starrocks::ThreadPool::dispatch_thread()
@ 0xe1b5085 starrocks::Thread::supervise_thread(void*)
@ 0x7f5ed47f3ac3 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x94ac2)
@ 0x7f5ed48858c0 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x1268bf)
[1776192387.223][thread: 140043098592832] je_mallctl execute purgeRocky
04/14/2026, 9:25 PMRewriteSimpleAggToMetaScanRule.
This rule attempts to speed up simple queries like SELECT COUNT(*) by reading metadata (MetaScan) instead of scanning the actual data. However, in version 4.0.3, there is a bug where this rewrite incorrectly handles nullable properties or column offsets (specifically in the UnionConstSourceOperator), leading to the SIGSEGV crash in the CN nodes you are seeing.
Why is this happening?
1. The Trigger: When you dropped and recreated the tables (especially changing from Primary Key to Duplicate Key), the metadata changed. The optimizer in 4.0.3 may be aggressively trying to use the "MetaScan" optimization for these new tables, but failing due to a mismatch in the internal column structures (the BinaryColumnBase::append error in your trace refers to the "offsets" of a string/binary column being handled incorrectly).
2. Global Impact: Because this is a global optimizer rule, once it's triggered, it often attempts to apply the same faulty logic to other tables in the same database or even different databases, which is why you are seeing crashes on tables you didn't even touch.
──────────
Immediate Workaround
You can disable this specific optimization rule globally. This will force StarRocks to perform a standard scan for COUNT(*) instead of using the problematic MetaScan path.
Run the following command as a user with administrative privileges:
sql
SET GLOBAL enable_rewrite_simple_agg_to_meta_scan = false;
Note: You may need to restart your CN nodes if they are stuck in a crash loop, but once this variable is set to false, the FE will stop generating the plan that causes the CN to crash.
──────────
Recommended Long-term Fix
This issue is addressed in later patch releases of the 4.0 branch. I strongly recommend upgrading to v4.0.5 or later.
* Multiple "MetaScan" and UnionConstSourceOperator stability fixes were included in 4.0.5.
* Version 4.0.3 has several known stability issues in shared-data mode that are resolved in more recent patches.
Additional Step (If crashes persist)
In a shared-data configuration, a crash can sometimes leave behind corrupted or partially written entries in the local disk cache (Data Cache). If the SET GLOBAL command above doesn't resolve the issue for existing tables:
1. Stop the CN nodes.
2. Clear the Data Cache directory on each CN node (usually located under your storage_root_path/datacache/).
3. Restart the CN nodes.
This ensures that any "poisoned" metadata or data blocks cached locally from the previous crashes are cleared.
References
• faq/shared_data_faq.md
• quick_start/shared-data.md
• deployment/shared_data/azure.md