when i use [starrocks-cluster-sync] tool to migrant,i got tons of ERRORs

[meta-handler] produceReplicationJob(ClusterMetaKeeper.java:253):
Failed to get source backend info for tablet xxx and backend id 10623495.
Detail: db->xxx, table->xxx, partition->xxx

Running

SHOW BACKENDS;

on the source cluster does not list

backend id 10623495

. This BE has been decommissioned. @Rocky

This error occurs because the starrocks-cluster-sync tool is attempting to locate a tablet replica on a Backend (BE) that no longer exists in the source cluster's active backend list, but is still referenced in the Frontend's (FE) tablet metadata. When a BE is decommissioned, the FE should migrate all its data and then remove the BE from the cluster metadata. If the migration tool encounters this error, it means the source FE is still returning a replica location for the old

backend id 10623495

, even though that BE is gone from

SHOW BACKENDS

. Recommended Steps to Resolve 1. Verify Metadata Inconsistency Identify one of the problematic tablets from your error logs and check its status on the source cluster:

sql
SHOW TABLET <tablet_id>;

Check if the output still lists a replica with

BackendId = 10623495

. Also, verify the health of the replicas:

sql
ADMIN SHOW REPLICA STATUS FROM <table_name> WHERE BackendId = 10623495;

2. Clean Up Stale Replicas Since the BE is already decommissioned and gone from the backend list, you can force the FE to drop these "ghost" replica records by marking them as bad. This will trigger the FE to remove the metadata entry and, if necessary, create a new replica on a healthy BE. For a specific tablet:

sql
ADMIN SET REPLICA STATUS PROPERTIES(
    "tablet_id" = "<tablet_id>", 
    "backend_id" = "10623495", 
    "status" = "bad"
);

3. Trigger a Table-wide Repair If you have a large number of errors, you can ask the FE to prioritize checking and repairing the metadata for the entire table:

sql
ADMIN REPAIR TABLE <db_name>.<table_name>;

Note: This command is a hint to the FE's Tablet Scheduler and may take some time to process depending on the cluster load. 4. Check for Unhealthy Tablets Check the overall health of your tablets to see if there are many "Inconsistent" or "Missing" replicas:

sql
SHOW PROC '/statistic';

If you see a high number of unhealthy tablets in the

InconsistentTabletNum

or

UnhealthyTabletNum

columns, follow the drill-down path (e.g.,

SHOW PROC '/statistic/<db_id>'

) to find the affected tablets. Why did this happen? * Partial Decommissioning: Sometimes the decommissioning process finishes its "safe" removal, but the background cleanup of tablet metadata (the "tablet report" process) has not yet cleared all stale entries from the FE memory. * Single Replica Tables: If a table was created with

replication_num = 1

, decommissioning a BE can be problematic if there isn't another BE available to take the data, though typically the decommission would hang in this case. * Metadata Bug: In some older versions, metadata regarding dropped backends could occasionally persist in the FE's tablet map until a restart or manual intervention. Summary: Mark the replicas on the non-existent BE as

bad

using

ADMIN SET REPLICA STATUS

. Once the metadata is clean and

SHOW TABLET

no longer mentions the old BE ID, the sync tool should proceed without errors. References • administration/management/BE_configuration.md • faq/Deploy_faq.md • faq/operation_maintenance_faq.md • administration/data_migration_tool.md • administration/management/resource_management/Replica.md

i wanna trigger a database-wide repair,what should i do,and will it stop service?

Hello, @202131221006! How can I assist you today?