Hey Team BE and FE pods not coming up Getting this in BE nod StarRocks #questions-and-troubleshooting

Hey Team, BE and FE pods not coming up. Getting th...

Ashutosh Gupta

11/05/2025, 3:30 PM

Hey Team, BE and FE pods not coming up. Getting this in BE nodes.

Copy code

[Wed Nov  5 20:50:02 IST 2025] Add myself (fcc-be-0.fcc-be-search.fcc.svc.cluster.local:9050) into FE ...
ERROR 1045 (28000): Access denied for user 'root' (using password: YES)
ERROR 1045 (28000): Access denied for user 'root' (using password: YES)
[Wed Nov  5 20:50:04 IST 2025] Add myself (fcc-be-0.fcc-be-search.fcc.svc.cluster.local:9050) into FE ...
ERROR 1045 (28000): Access denied for user 'root' (using password: YES)
ERROR 1045 (28000): Access denied for user 'root' (using password: YES)

We changed the creds of root user, 1 month before and implement using

initPassword

. Kube pods restarted due to some issue, now pods are not coming up. Helm chart

Copy code

initPassword:
    enabled: true
    
    isInstall: false
    password: ""
   
    passwordSecret: "kube-starrocks-v1-fcc-secret"

Secret file ,

kube-starrocks-v1-fcc-secret

has the password of root user. Needed help 🙏

Kevin Cai

11/05/2025, 3:59 PM

try to describe the pod, it should have an env var named

MYSQL_PWD

injected from the secret, make sure it has the right password of the root user.

Ashutosh Gupta

11/05/2025, 4:26 PM

@Kevin Cai as this helm chart is not running for the first time and password is already set , made

initPassword.enabled

false, then pods is starting up. Is it the correct flow, once password set, second time onwards make it false ?

Ashutosh Gupta

11/06/2025, 1:57 AM

All backend nodes got down

Copy code

BackendId: 12494 IP: fcc-be-1.fcc-be-search.fcc.svc.cluster.local HeartbeatPort: 9050 BePort: -1 HttpPort: -1 BrpcPort: -1 LastStartTime: NULL LastHeartbeat: NULL Alive: false SystemDecommissioned: false ClusterDecommissioned: false TabletNum: 0 DataUsedCapacity: 0.000 B AvailCapacity: 1.000 B TotalCapacity: 0.000 B UsedPct: 0.00 % MaxDiskUsedPct: 0.00 % ErrMsg: Unmatched cluster id Version: Status: {"lastSuccessReportTabletsTime":"N/A"} DataTotalCapacity: 0.000 B DataUsedPct: 0.00 % CpuCores: 0 MemLimit: 0.000B NumRunningQueries: 0 MemUsedPct: 0.00 % CpuUsedPct: 0.0 % DataCacheMetrics: N/A Location: StatusCode: DISCONNECTED

Ashutosh Gupta

11/06/2025, 2:19 AM

Fix this using deleting cluster-id. @Kevin Cai but seeing data loss, if I change the cluster name, will it impact data loss ?

Kevin Cai

11/06/2025, 2:27 AM

are you provided pvc configuration for the fe/meta directory?

Ashutosh Gupta

11/06/2025, 2:32 AM

Didn't get you. we are using helm chart

Copy code

root@fcc-fe-0:/opt/starrocks/fe/meta# ls
bdb  image  lost+found

Do we need to explicitly specify the PVC configurations?

Kevin Cai

11/06/2025, 2:39 AM

this part is important, otherwise your fe meta will be stored on emptydir which will be lost when pod restart.

Ashutosh Gupta

11/06/2025, 2:42 AM

Copy code

storageSpec:
      
      name: "fcc-starrocks-fe"
      
      storageSize: 50Gi
      
      storageMountPath: ""
      
      logStorageSize: 10Gi

it is defined

Kevin Cai

11/06/2025, 3:20 AM

what's the default storageclass for the pvc?

Ashutosh Gupta

11/06/2025, 3:22 AM

it is empty

Copy code

storageClassName: ""

Ashutosh Gupta

11/06/2025, 3:31 AM

it is retain

Open in Slack

Previous Next