@Rocky can I lower fespec replicas after increasing

Yes, you can lower the number of FE replicas in your

starRocksFeSpec

after increasing them, but there are a few important considerations and manual steps required to ensure the cluster remains healthy. 1. Update the Replica Count If you are using the StarRocks Operator, you can decrease the

replicas

field in your StarRocksCluster YAML:

yaml
spec:
  starRocksFeSpec:
    replicas: 3 # For example, lowering from 5 to 3

The Operator will then terminate the extra FE Pods (starting from the highest index). 2. Manual Cleanup (Critical) Simply reducing the replica count in Kubernetes does not automatically remove those FE nodes from the StarRocks internal metadata. If you don't perform this step, the cluster will still try to communicate with the deleted Pods, and they will appear as

Alive: false

in your node list. To clean up, log in to your StarRocks cluster (via MySQL client) and follow these steps: 1. Check the status of FE nodes:

sql
    SHOW PROC '/frontends';

Identify the IPs/Hostnames of the FE nodes that were terminated (those where

Alive

is

false

). 2. Drop the removed nodes: Depending on whether the removed nodes were Followers or Observers, run:

sql
    -- If it was a Follower
    ALTER SYSTEM DROP FOLLOWER "fe_host:edit_log_port";

    -- If it was an Observer
    ALTER SYSTEM DROP OBSERVER "fe_host:edit_log_port";

_Note: The

edit_log_port

is typically

9010

by default._ 3. Best Practices & Constraints * Odd Number of Followers: For High Availability (HA), it is strongly recommended to keep an odd number of Follower FE nodes (e.g., 1, 3, or 5). This is required for the BDB JE voting mechanism to reach a quorum. Scaling from 3 down to 2 is not recommended; it's better to scale from 3 to 1 if you are reducing resources. * Leader Node: Avoid dropping the current Leader node directly if possible. If you must scale down and the Leader is one of the nodes being removed, the cluster will trigger a new election among the remaining Followers. * Data Integrity: Scaling FE nodes only affects the management and metadata layer; it does not trigger data redistribution like BE scaling does. However, ensuring a healthy FE quorum is vital for cluster stability. In summary: Yes, you can lower them, but you must manually run

ALTER SYSTEM DROP

for each removed node to keep the metadata clean. References • deployment/sr_operator.md • administration/stargo.md • administration/management/Scale_up_down.md • faq/operation_maintenance_faq.md • faq/Deploy_faq.md • deployment/downgrade.md

@Rocky there was an error downsizing am I able to downsize from 3 to 1 fespecs?

Yes, you can downsize from 3 to 1 FE node, but it requires a specific procedure. If you simply change the replica count from 3 to 1 (e.g., in your

starRocksFeSpec

), the cluster will likely lose quorum and stop functioning. Why the error happens StarRocks FEs use a consensus protocol (BDB JE) to manage metadata. In a 3-node setup, a majority (at least 2 nodes) must be alive for the cluster to elect a Leader and remain operational. * If you suddenly remove 2 nodes, the remaining 1 node cannot form a majority of the original 3. * Consequently, the remaining FE will stay in a

FOLLOWER

or

UNKNOWN

state and will not accept connections or process queries. ────────── Correct Procedure (StarRocks Operator) If you are using the StarRocks Operator (indicated by your use of

fespecs

), follow these steps to safely downsize: 1. Drop the FE nodes logically first: Connect to your cluster via a MySQL client and manually remove two of the Follower nodes one by one.

sql
    -- Check your FE list first
    SHOW PROC '/frontends';

    -- Drop a follower (replace with actual IP and EditLogPort, usually 9010)
    ALTER SYSTEM DROP FOLLOWER "fe_ip_1:9010";

    -- Wait a moment, check SHOW PROC again, then drop the second one
    ALTER SYSTEM DROP FOLLOWER "fe_ip_2:9010";

2. Update the Spec: Only after the nodes are removed from the StarRocks metadata and only 1 node remains in

SHOW PROC '/frontends'

, should you update your Kubernetes

starRocksFeSpec

to

replicas: 1

. ────────── How to Recover if the Cluster is Already Stuck If you have already downsized and the single FE is unreachable (stuck without a leader), you need to force the remaining FE to become the Leader by resetting the election group: 1. Edit the FE Configuration: You need to add

bdbje_reset_election_group = true

to the

fe.conf

of the remaining FE. 2. Restart the FE: Once the FE starts up, it will ignore the old 3-node configuration and elect itself as the Leader of a new 1-node group. 3. Cleanup: After it successfully becomes the Leader, remove that line from

fe.conf

and restart it one more time to return to normal operation. Note for Operator users: Since the Operator manages the config, you may need to add this parameter via the

configMap

or

feEnvVars

temporarily, or check the specific Operator version documentation for "metadata recovery" steps. Important Recommendation For any production environment, it is highly recommended to maintain 3 FE Follower nodes to ensure High Availability (HA). A 1-FE cluster is a single point of failure; if that node or its disk fails, your entire database becomes unavailable and metadata recovery may be required. References • administration/Meta_recovery.md • deployment/deploy_manually.md