<@U0A71G31CDV> while downscaling the cluster size,...
# questions-and-troubleshooting
k
@Rocky while downscaling the cluster size, all the nodes have become follower and there is no leader currently. getting the following error:
Copy code
ERROR 1064 (HY000): (JE 18.3.20) Could not determine master from helpers
r
It sounds like your FE (FrontEnd) cluster has lost its quorum during the downscaling process. This error typically occurs when the remaining FE nodes cannot reach a majority to elect a new Leader, often because the metadata still expects the nodes you removed to be part of the election group. To recover from this state, you need to force one of the remaining Follower nodes to become the Leader and reset the election group. Recovery Steps (The "Measure of Last Resort") Follow these steps on one of your remaining FE Follower nodes (preferably the one with the most up-to-date metadata): 1. Stop all FE nodes in the cluster. 2. Modify the configuration of the chosen FE node: Edit
fe/conf/fe.conf
and add the following line:
Copy code
properties
    bdbje_reset_election_group = true
3. Restart this FE node:
Copy code
bash
    ./fe/bin/start_fe.sh --daemon
4. Verify the status: Log into this FE node via MySQL client and run:
Copy code
sql
    SHOW FRONTENDS;
Confirm that this node has the role
LEADER
and
Alive
is
true
. 5. Clean up the configuration: Once the node is successfully running as the Leader, remove the
bdbje_reset_election_group = true
line from
fe.conf
. You should restart this node one more time without that flag to ensure it's running in a normal state. 6. Re-add other nodes: For the other FE nodes you want to keep: * Stop the FE process. * Crucial: Clear their metadata directory (the path defined by
meta_dir
in
fe.conf
, default is
fe/meta
). * Restart them using the
--helper
flag pointing to the new Leader:
Copy code
bash
        ./fe/bin/start_fe.sh --helper <leader_ip>:<edit_log_port> --daemon
Why did this happen? In StarRocks, FE Followers use a protocol (BDBJE) that requires a majority (quorum) to elect a leader. If you had 3 followers and removed 2 without the cluster correctly updating its internal voting list, the remaining 1 node cannot form a majority (which would be 2 out of 3) and thus cannot become a Leader. Note: For future downscaling, ensure you use
ALTER SYSTEM DROP FOLLOWER "ip:port";
while the cluster is still healthy so the metadata is updated correctly before nodes are taken offline. Referencesadministration/Meta_recovery.mdfaq/Deploy_faq.md