@Rocky I am seeing this error after having moved a table to 128 buckets to put it in a group, is this a known issue? {"@timestamp":"2026-07-02 120137.301Z","level":"ERROR","thread.name":"colocate-group-clone-checker","thread.id":202,"line":100,"file":"Daemon.java","method":"run","message":"daemon thread got exception. name: colocate-group-clone-checker","exception":"java.lang.IllegalStateException: 128 v.s. 16\n\tat com.google.common.base.Preconditions.checkState(Preconditions.java:512)\n\tat com.starrocks.clone.ColocateTableBalancer.doMatchOneGroup(ColocateTableBalancer.java:798)\n\tat com.starrocks.clone.ColocateTableBalancer.matchOneGroupNonUrgent(ColocateTableBalancer.java:917)\n\tat com.starrocks.clone.ColocateTableBalancer.matchGroups(ColocateTableBalancer.java:938)\n\tat com.starrocks.clone.ColocateTableBalancer.runAfterCatalogReady(ColocateTableBalancer.java:259)\n\tat com.starrocks.common.util.FrontendDaemon.runOneCycle(FrontendDaemon.java:78)\n\tat com.starrocks.common.util.Daemon.run(Daemon.java:98)\n","filepath":"/var/log/app/fe.log"}

This error is a result of a bucket count mismatch within a Colocation Group. In StarRocks, all tables in the same Colocation Group must have the exact same number of buckets and the same distribution column types. The exception

java.lang.IllegalStateException: 128 v.s. 16

indicates that the background daemon (

ColocateTableBalancer

) found a table with 128 buckets assigned to a group that is defined with 16 buckets (or vice-versa). Why this happened When you "moved the table to 128 buckets," if that table was joined to an existing Colocation Group that was already established with 16 buckets, the metadata became inconsistent. StarRocks usually prevents this during the

ALTER TABLE

or

CREATE TABLE

stage, but inconsistencies can occasionally occur if the group properties were modified in a specific sequence or if the group was empty but still held old metadata. How to diagnose Run the following command to check the current state and schema of your Colocation Groups:

sql
SHOW PROC '/colocation_group';

Look for your group name and check the

BucketsNum

column. It will likely show either 16 or 128. If the group shows 16, but your table has 128, the balancer will fail with the error you saw. To see which tables are in that group and their status: 1. Find the

GroupId

from the previous command (e.g.,

10005.10008

). 2. Run:

SHOW PROC '/colocation_group/<GroupId>';

How to fix To resolve the mismatch, all tables in the group must be aligned to the same bucket count. You cannot modify the bucket count of a single table while it is part of a Colocation Group. Use the following procedure: 1. Remove the table(s) from the group:

sql
    ALTER TABLE <table_name> SET ("colocate_with" = "");

2. Verify/Fix Bucket Counts: Ensure all tables you intend to group together have the desired number of buckets (e.g., 128). If you need to change a table's bucket count, you may need to recreate the table or use a swap procedure if it's an OLAP table, as the distribution bucket count is fixed at partition creation. 3. Re-add to the group: Once the bucket counts match the group's intended schema:

sql
    ALTER TABLE <table_name> SET ("colocate_with" = "<group_name>");

Note: If the group itself is stuck with the wrong metadata (e.g., it thinks it should be 16 buckets but you want 128), remove all tables from that group. Once the last table is removed, the group is deleted. Re-adding the first table with 128 buckets will recreate the group with the correct 128-bucket schema. References • using_starrocks/Colocate_join.md

@Rocky the group shows the right buckets, 128

If

SHOW PROC '/colocation_group';

shows the group is set to 128 buckets, then the error

128 v.s. 16

confirms that at least one table assigned to that group still has 16 buckets in its metadata. This is a known behavior when a table is added to a Colocation Group with a bucket count that doesn't match the table's existing physical structure. StarRocks requires all tables in a group to have the exact same number of buckets across all partitions. 1. Identify the Mismatched Table The error is being thrown because the background balancer is iterating through the tables in your group and found one that doesn't match. You can find the culprit by running:

sql
-- Replace <group_id> with the ID found in SHOW PROC '/colocation_group'
SHOW PROC '/colocation_group/<group_id>';

In the output, look at the list of tables. Check the bucket count for each table. One of them will likely still show 16. 2. Why the mismatch exists In StarRocks, the number of buckets is defined at the partition level. If you had an existing table with 16 buckets and tried to "move" it to 128 buckets by changing the table property, this does not retroactively change existing partitions. If you used

ALTER TABLE ... SET ("colocate_with" = "group_name")

on a table that physically has 16-bucket partitions, the property might have been accepted, but the background

colocate-group-clone-checker

(which ensures data is physically aligned) is now failing because it sees the physical 16-bucket tablets. 3. How to Resolve To fix the

IllegalStateException

and restore the background balancer: 1. Remove the problematic table from the group:

sql
    ALTER TABLE <mismatched_table_name> SET ("colocate_with" = "");

This should stop the FE error logs immediately. 2. Realign the bucket count: Since you cannot change the bucket count of existing partitions, you must recreate the table with 128 buckets or use the

ALTER TABLE ... MODIFY DISTRIBUTION

(if supported for your table type) which typically requires a full data redistribution. * Recommended approach: Create a new table with

DISTRIBUTED BY HASH(...) BUCKETS 128

, insert the data from the old table, and then add it to the Colocation Group. 3. Add back to the group: Only after the table physically has 128 buckets, run:

sql
    ALTER TABLE <table_name> SET ("colocate_with" = "your_group_name");

Note: If the FE continues to crash or show this error after removing the table, it may be necessary to restart the Leader FE to clear the inconsistent state from the `ColocateTableBalancer`'s memory, though removing the table from the group is usually sufficient. References • using_starrocks/Colocate_join.md • best_practices/bucketing.md