@Rocky After upgrading to StarRocks 4.1.0, my Compute Node is crashing with a SEGFAULT on startup.

[Wed Apr 15 10:21:01 KST 2026] Process conf file cn.conf ...
[Wed Apr 15 10:21:01 KST 2026] Add myself (kitty-starrocks-cn-4.kitty-starrocks-cn-search.kitty-starrocks.svc.cluster.local:9050) into FE ...
ERROR 1064 (HY000) at line 1: Compute node already exists with same host kitty-starrocks-cn-4.kitty-starrocks-cn-search.kitty-starrocks.svc.cluster.local and port 9050
[Wed Apr 15 10:21:01 KST 2026] run start_cn.sh
/opt/starrocks/cn_entrypoint.sh: line 165:    27 Segmentation fault      (core dumped) $STARROCKS_HOME/bin/start_cn.sh $addition_args

Hi there! I'm Rocky. It looks like you've run into a critical issue after upgrading to StarRocks 4.1.0. Based on the logs you provided, the

Segmentation fault

is occurring during the execution of the

starrocks_be

binary (which runs as the Compute Node process). 1. The "Already Exists" Error is a Red Herring First, don't worry about the

ERROR 1064 (HY000)

message. This occurs when the Kubernetes entrypoint script tries to register the CN with the FE. Since you are upgrading an existing cluster, the node is already registered. The script correctly ignores this error and proceeds to launch the binary. 2. Primary Cause: Cache Incompatibility The most common cause for a SEGFAULT on startup after a major upgrade (especially to 4.1.x) is an incompatible local data cache. StarRocks 4.1.0 introduces significant internal changes to the shared-data architecture, including automatic tablet splitting and range-based data distribution. If your CN has a persistent volume mounted to its

storage_root_path

(default:

${STARROCKS_HOME}/storage

), the binary may be crashing while trying to load the old metadata/cache files from previous versions. Solution: Clean the Local Cache Since Compute Nodes (CNs) are stateless (all permanent data is in your object storage like S3/HDFS), it is safe to wipe the local cache. * In Kubernetes: If you are using a

StatefulSet

with a

volumeClaimTemplate

for storage, you should temporarily clear the contents of that volume or recreate the pods without the old data. * Manually: Delete the contents of the

storage

directory on the node:

bash
    rm -rf /opt/starrocks/storage/*

3. Check Hardware Compatibility (AVX2) StarRocks 4.1.0 and recent 3.x versions require AVX2 instruction support on the CPU. If you are running on older hardware or a virtualized environment that doesn't pass through AVX2 instructions, the binary will SEGFAULT immediately. * Verify support on your node:

bash
    cat /proc/cpuinfo | grep avx2

If no results appear, you may need to use a "no-AVX2" build if provided, or upgrade your instance type. 4. Inspect Detailed Logs To get the exact stack trace of the crash, check the following file inside the container: *

$STARROCKS_HOME/log/cn.out

*

$STARROCKS_HOME/log/cn.WARNING

If you see a stack trace involving

starrocks::lake::TabletManager

or

starlet

, it confirms the cache/metadata initialization issue. Recommendation: Try clearing the

storage_root_path

first. This solves the vast majority of "crash on startup" issues following a 4.1 upgrade in shared-data environments. If the issue persists, please check the

cn.out

log for the specific stack trace! References • deployment/upgrade.md • administration/management/BE_configuration.md • deployment/downgrade.md • deployment/shared_data/minio.md • administration/Meta_recovery.md • deployment/shared_data/azure.md • deployment/shared_data/hdfs.md • faq/Deploy_faq.md

@Rocky I've ruled out cache issues by using a fresh PVC, and there are no hardware problems either.

Since you have already ruled out hardware (AVX2) and local cache corruption (via a fresh PVC), a SEGFAULT on startup in StarRocks 4.1.0 is often tied to environment-level mismatches or specific architectural changes in the 4.1.x release. Here are the most likely causes and how to troubleshoot them: 1. The "Lake Service" and

storage_root_path

In StarRocks 4.1.0, the shared-data architecture (which uses Compute Nodes) relies heavily on the Lake Service and Starlet for tablet management and the new automatic tablet splitting. * The Issue: Even though CNs are stateless, they require a writable

storage_root_path

to initialize their internal metadata and temporary buffers. If this path is missing, read-only, or not explicitly defined in your

cn.conf

, the binary may crash during the initialization of the Lake Service. * Fix: Ensure your

cn.conf

explicitly contains:

conf
    storage_root_path = /opt/starrocks/storage

And verify that your PVC is indeed mounted to that exact path with read/write permissions for the user running the process. 2. Port Conflicts (Wait for

TIME_WAIT

) The log shows the CN is attempting to register with port 9050. Because this is a restart/upgrade, the FE already knows about this host. * The Issue: If the previous CN process crashed or was terminated, the TCP ports (9050, 9060, 8040, 8060) might still be in a

TIME_WAIT

state in the network stack. If the new 4.1.0 binary attempts to bind to these ports and fails to handle the

EADDRINUSE

error gracefully during its early boot sequence, it can trigger a SEGFAULT. * Fix: Check if there are any zombie processes in your pod or wait a few minutes before the next restart to allow ports to clear. 3. Starlet Port Configuration StarRocks 4.1.0 introduces more rigorous checks for the Starlet Agent (the component managing remote storage). * The Issue: If your

cn.conf

is missing the

starlet_port

(default 9070) or if it conflicts with another service, the CN might crash on startup. * Fix: Ensure

starlet_port = 9070

is defined in

cn.conf

and that this port is exposed/available in your K8s service/pod definition. 4. Memory/CGroup Limits in Kubernetes StarRocks 4.1.0 has updated its memory management to support more concurrent tablet operations. * The Issue: If your Kubernetes

resources.limits.memory

is too close to the

mem_limit

configured in

cn.conf

(which defaults to 90%), the binary may crash while trying to allocate its initial memory pools (like the block cache or metadata cache). * Fix: Try lowering the memory limit in

cn.conf

to see if it stabilizes:

conf
    mem_limit = 80%

5. Detailed Stack Trace (The "True" Answer) The

Segmentation fault

in your log is caught by the shell script, but the stack trace will be inside the CN's own log files. To find the exact reason: 1. Exec into the container (if it stays up long enough) or use

kubectl cp

to grab the logs. 2. Check:

/opt/starrocks/log/cn.out

3. Look for a block starting with

*_ SIGSEGV _*

or

*_ Aborted at ... _*

. Common 4.1.0 Stack Traces: * If you see

starrocks::lake::TabletManager

: It's a metadata/storage path issue. * If you see

starlet::...

: It's a configuration issue with your remote storage (S3/HDFS) or

starlet_port

. * If you see

jvm

or

jni

: It's a Java environment issue. Ensure the image's

JAVA_HOME

is correct. Next Step: Could you check the contents of

/opt/starrocks/log/cn.out

? The stack trace there will point directly to the function that is failing. References • administration/Meta_recovery.md • developers/build-starrocks/Build_in_docker.md • deployment/deploy_manually.md