Hey team! I had a quick question about Cluster Sn...
# questions-and-troubleshooting
m
Hey team! I had a quick question about Cluster Snapshot and garbage collection, especially as we think about a backup strategy to protect against any potential data corruption on the FE: Here’s how I currently understand it: • Data objects are protected from GC if they’re referenced by either the current cluster state or a snapshot. • So, if I deleted a partition at 10:05, the snapshot from 10:00 would still keep those files safe from GC. • Then, when a new snapshot is created at 10:10 (and the old one is dropped), those files could become eligible for GC. Am I getting this right, or does GC only look at the current cluster state and ignore snapshots? Just want to make sure our backup approach actually keeps the data safe in case anything goes wrong on the FE. Thanks a bunch! 🙏
k
yes, GC respects cluster snapshot references.
🙏 1
m
Hey @Kevin Cai, I hope you are doing well. We ran into an issue while following the DR activity steps from this README (step 5): https://github.com/StarRocks/starrocks-kubernetes-operator/blob/main/doc/disaster_recovery_for_shared_data_cluster_howto.md Here’s the FE error log we encountered:
Copy code
[Tue Oct 28 12:43:36 UTC 2025] Process conf file fe.conf..
[Tue Oct 28 12:43:36 UTC 2025] first start fe with meta not exist
[Tue Oct 28 12:43:36 UTC 2025] FE service kube-starrocks-fe-service.kube-starrocks:9030 is not alive yet!
[Tue Oct 28 12:43:36 UTC 2025] FE service kube-starrocks-fe-service.kube-starrocks:9030 is not alive yet!
[Tue Oct 28 12:43:36 UTC 2025] FE service kube-starrocks-fe-service.kube-starrocks:9030 is not alive yet!
[Tue Oct 28 12:43:36 UTC 2025] FE service kube-starrocks-fe-service.kube-starrocks:9030 is not alive yet!
[Tue Oct 28 12:43:36 UTC 2025] FE service kube-starrocks-fe-service.kube-starrocks:9030 is not alive yet!
[Tue Oct 28 12:43:36 UTC 2025] FE service kube-starrocks-fe-service.kube-starrocks:9030 is not alive yet!
[Tue Oct 28 12:44:06 UTC 2025] Timed out, no members detected ever, assume myself is the first node..
[Tue Oct 28 12:44:06 UTC 2025] first start with no meta run start_fe.sh with additional options: '—host_type FQDN'
We’re using Helm and StarRocks version 4.0-latest. Would love to get your feedback on this 🙏 Thanks!
k
these are regular logs, indicating the start of the first FE node.
m
For some reason, after the last log, the fe pod gets restarted (and goes to infinite loop)
k
how much disk space allocated for fe meta volume?
m
200GB
We are restoring the snapshot
k
how about set 'LOG_CONSOLE = 1' so the app log will be redirected to pod console, will provide more info to troubleshoot. https://github.com/StarRocks/starrocks-kubernetes-operator/blob/main/doc/logging_and_related_configurations_howto.md#4-logging-to-the-console
👍 1