@Rocky I understand that to increase PVC volume size in StarRocks, I need to directly patch the PVC via kubectl, then delete the StatefulSet with --cascade=orphan and let the Operator recreate it. Are there any side effects from the StarRocks Operator when doing this?

Resizing the PVC volume for a StarRocks cluster managed by the Operator using the

--cascade=orphan

method is a common practice because the

volumeClaimTemplates

field in a Kubernetes StatefulSet is immutable. While this is an effective workaround, there are a few side effects and behaviors from the StarRocks Operator you should be aware of: 1. Rolling Restart of Pods Even though you use

--cascade=orphan

to keep the Pods running while the StatefulSet is deleted, once the StarRocks Operator recreates the StatefulSet with the new storage configuration, the StatefulSet controller will notice a discrepancy between the running Pods (created by the old spec) and the new spec. * Effect: This will trigger a rolling restart of the FE or BE Pods to bring them into compliance with the new StatefulSet definition. * Risk: StarRocks will remain available during a rolling restart (provided you have enough replicas), but you should ensure the cluster is healthy before starting the process. 2. Status Inconsistency During the window between deleting the StatefulSet and the Operator recreating it: * Effect: The Operator's reconciliation loop might report errors or an "Unhealthy" status for the

StarRocksCluster

custom resource because it cannot find the underlying StatefulSet. * Note: This is temporary and resolves once the Operator successfully recreates the resource. 3. CRD vs. StatefulSet Synchronization If you patch the PVC and the StatefulSet manually but forget to update the

storageSize

in your

StarRocksCluster

CRD, the Operator will eventually try to "revert" the StatefulSet to the size defined in the CRD. * Recommendation: Always update the

storageSize

in your

StarRocksCluster

YAML/spec to match the new patched PVC size before or immediately after the orphan deletion. 4. Data Safety * Effect: There is no data loss inherent to this process. Because the PVCs are not deleted and the Pods are orphaned, the StarRocks data (metadata in FE or tablets in BE) remains intact on the disks. Recommended Workflow Summary To minimize side effects, follow this sequence: 1. Check StorageClass: Ensure

allowVolumeExpansion: true

is set in your StorageClass. 2. Patch PVCs:

kubectl patch pvc <pvc-name> -p '{"spec":{"resources":{"requests":{"storage":"<new-size>"}}}}'

. 3. Update CRD: Edit your

StarRocksCluster

resource to reflect the new

storageSize

. 4. Orphan Delete:

kubectl delete sts <sts-name> --cascade=orphan

. 5. Reconcile: The Operator will see the StatefulSet is missing and recreate it using the new size from the CRD. The StatefulSet will then "adopt" the existing Pods and PVCs and begin a rolling update. Note: Always monitor the progress of the volume expansion via

kubectl describe pvc <pvc-name>

. Some storage backends require the Pod to be restarted (which happens in step 5) before the file system actually expands. References • quick_start/helm.md • deployment/sr_operator.md

@Rocky Even if the StatefulSet's volumeClaimTemplates are recreated with the same disk size as the patched StarRocksCluster CR spec, does a rolling restart of the pods still occur?

The behavior of the StarRocks Operator regarding rolling restarts when resizing disks depends on how the Operator reconciles the state between the

StarRocksCluster

Custom Resource (CR) and the underlying

StatefulSet

. Here is the breakdown of why a rolling restart is highly likely to occur, even if you have manually aligned the `volumeClaimTemplates`: 1. Hash Mismatch Triggers Reconciliation The StarRocks Operator typically uses a hash annotation (stored on the

StatefulSet

) to determine if the current

StatefulSet

configuration matches the desired state defined in the

StarRocksCluster

CR. * When you patch the

StarRocksCluster

CR (e.g., changing

storageSize

), the Operator calculates a new hash for that component (FE/BE/CN). * Because the hash in the

StatefulSet

metadata is now "old" compared to the new CR spec, the Operator will trigger a reconciliation to update the

StatefulSet

. 2. Immutability of

volumeClaimTemplates

In Kubernetes, the

volumeClaimTemplates

field of a

StatefulSet

is immutable. * To apply the new disk size to the

StatefulSet

definition, the Operator must delete the existing

StatefulSet

(usually with the

Orphan

propagation policy to keep Pods running) and recreate it with the updated template. * Even if the physical disk size in your manual template and the CR spec match, the act of the Operator "re-syncing" the

StatefulSet

to update its metadata (like the hash) often results in the

StatefulSet

controller observing a "new revision." 3. StatefulSet Controller Logic Even if the

StatefulSet

is successfully updated/recreated to match your manual configuration: * If the

PodTemplateSpec

(the part defining the container, env vars, etc.) is identical to the current running Pods, the standard Kubernetes

StatefulSet

controller might not restart them. * However, the StarRocks Operator often updates annotations or labels during this process. If any part of the Pod template is modified (including metadata), the

StatefulSet

controller will initiate a rolling restart to bring all Pods to the "current" revision. Recommendation If your goal is to expand disks without downtime: 1. Expand PVCs first: Manually update the PVCs (if your StorageClass supports

allowVolumeExpansion

). 2. Update the CR: Update the

StarRocksCluster

CR to reflect the new size so that the Operator stays in sync. 3. Expect the Restart: Plan for a rolling restart. The Operator is designed to perform this restart gracefully (one node at a time), ensuring the cluster remains available and data remains consistent (e.g., waiting for tablet migration if using BEs). If you want to minimize the impact of the restart, ensure Graceful Exit is enabled (available from v3.3 onwards), which allows the StarRocks nodes to finish in-flight queries before shutting down during the rolling update. References • using_starrocks/Colocate_join.md • administration/management/graceful_exit.md • administration/management/resource_management/Replica.md • deployment/sr_operator.md