Hi @Rocky I'm hitting a persistent deadlock issue when deploying a single-node FE cluster on Kubernetes using the StarRocks Helm Chart/Operator. The Behavior: • If I deploy without a PVC (

emptyDir

), the FE pod boots up and runs perfectly fine. • The moment I attach a persistent PVC (Longhorn StorageClass), the FE pod goes into a crash loop during initialization. Logs:

[Tue Jul 14 13:36:08 CST 2026] Process conf file fe.conf ...                                                         │
│ [Tue Jul 14 13:36:08 CST 2026] first start fe with meta not exist.                                                   │
│ [Tue Jul 14 13:36:09 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:12 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:15 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:18 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:21 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:24 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:27 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:30 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:34 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:37 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:40 CST 2026] FE service starrocks-fe-service.starrocks:9030 is not alive yet!                      │
│ [Tue Jul 14 13:36:40 CST 2026] Timed out, no members detected ever, assume myself is the first node ..               │
│ [Tue Jul 14 13:36:40 CST 2026] first start with no meta run start_fe.sh with additional options: ' --host_type FQDN'

I have also tried added, but no improvements found:

runAsNonRoot: true
    
podSecurityContext:
  fsGroup: 1000

I will also attach the Helm values file in this thread.

Based on your logs and the behavior described (working with

emptyDir

but failing with Longhorn PVC), this is a common issue related to filesystem permissions or FQDN resolution timing during the initial metadata creation. The log

FE service ... is not alive yet!

followed by the timeout is expected for a single-node cluster (it’s checking if a leader already exists). The crash happens right after it attempts to execute

start_fe.sh

. 1. File Permissions (Most Likely) Even with

fsGroup: 1000

, some CSI drivers (like Longhorn) might not automatically apply permissions to the root of the volume if the mount point already exists or if the container is running as a non-root user that lacks the authority to change the ownership of the mount. StarRocks FE needs write access to the

meta

and

log

directories. If it cannot create the initial BDB (Berkeley DB) files, the Java process will exit immediately. Solution: Add an

initContainer

to your Helm values to explicitly set the permissions on the mounted volume.

yaml
starRocksFeSpec:
  initContainers:
    - name: volume-permission
      image: busybox
      command: ['sh', '-c', 'chown -R 1000:1000 /opt/starrocks/fe/meta /opt/starrocks/fe/log']
      volumeMounts:
        - name: fe-meta
          mountPath: /opt/starrocks/fe/meta
        - name: fe-log
          mountPath: /opt/starrocks/fe/log
      securityContext:
        runAsUser: 0 # Run as root to fix permissions

2. FQDN Resolution Deadlock Your logs show the FE is starting with

--host_type FQDN

. In a Kubernetes environment: 1. The FE tries to bind to its FQDN (e.g.,

starrocks-fe-0.starrocks-fe-service...

). 2. If the Headless Service or the Pod's DNS record is not yet resolvable by the Pod itself, the Java process will throw an

UnknownHostException

and exit. 3. This is more common with persistent storage because the volume attachment process can sometimes delay the network stack's readiness relative to the entrypoint script execution. Check: Verify if the FE can resolve its own FQDN. If the FE process exits, check the

fe.log

or

fe.out

(which might be inside the PVC or visible via

kubectl logs <pod> --previous

). 3. Longhorn Specific: Synchronous Write/Locking StarRocks' metadata engine (BDB) is sensitive to disk latency and locking. If Longhorn is under heavy load or if there are stale locks from a previous failed mount, BDB might fail to initialize. • Check the Longhorn UI to ensure the volume is successfully "Attached" and "Healthy." • If you see

Locking

errors in the FE logs, you may need to manually clear the

meta/image/

directory if it contains corrupted/empty files from a failed start. Recommended Troubleshooting Steps: 1. Inspect the FE log: Since the entrypoint script completes, the error is inside the Java process. Run:

bash
    kubectl logs <fe-pod-name> -p

Look for

java.io.IOException: Permission denied

or

java.net.UnknownHostException

. 2. Verify the UID: Ensure the user ID inside the StarRocks image matches your

fsGroup

. Most official images use UID

1000

or

root

. 3. Temporary Test: Try setting

privileged: true

in the container's

securityContext

temporarily to rule out permission/mount issues. If you can provide the output of

kubectl logs <pod-name> --previous

or the content of

fe.warn.log

from the PVC, I can give you a more specific fix. References • administration/management/enable_fqdn.md • deployment/environment_configurations.md • faq/Deploy_faq.md

values.yaml

FE storage size too small, FE meta requires at least 5G free space, so you have to provide a PVC with more free space, suggest at least start with 10G

Thanks for pin pointing the issue @Kevin Cai