@Rocky I’m running StarRocks in shared-data mode (version: 4.1.1). After a cluster restart, tablet primaries ended up concentrated on a subset of backend nodes rather than spread evenly. Dropping and recreating the table with identical data produced a noticeably more even distribution. Three things I’d like to understand: 1. Is there an operator-facing way to redistribute tablet primaries on an existing table — an ADMIN command, a rebalance job, or a config knob — without dropping and recreating it? 2. What causes the post-restart skew? Is primary/leader assignment deterministic, or is it driven by node registration order, heartbeat timing, or similar recovery-time factors? 3. Why did recreating the table improve distribution? Is that simply fresh placement at creation time, or does a different balancing path apply?

In StarRocks shared-data mode (Cloud-native), "redistributing tablet primaries" refers to reassigning which Backend (BE) or Compute Node (CN) is responsible for a tablet’s metadata, compaction tasks, and primary data caching. 1. How to redistribute tablet primaries on an existing table In shared-data mode, tablet assignment is managed automatically by the Tablet Scheduler in the Frontend (FE). There is no explicit

ADMIN

command to manually "kick" a specific table's tablets to new nodes, but you can control and trigger the process through configurations: * The Global Switch: Ensure

lake_enable_balance_tablets_between_workers

is set to

true

(it is the default since v3.3.4). This allows the FE to migrate tablet "ownership" between nodes to reach a balanced state.

sql
    ADMIN SET FRONTEND CONFIG ("lake_enable_balance_tablets_between_workers" = "true");

* Triggering a Balance: The scheduler typically runs every minute. If you want to force it to reconsider assignments, you can briefly toggle the balance off and on, though usually, the system will naturally converge if the nodes are healthy. * Monitoring Status: You can check the current progress of the balancer using:

sql
    SHOW PROC '/cluster_balance/balance_stat';

If "inter-node tablet distribution" shows

Balanced: false

, the system is aware of the skew and is working to fix it. 2. What causes the post-restart skew? Post-restart skew is almost always driven by Node Registration Order and Heartbeat Timing. * Opportunistic Assignment: When a cluster restarts, nodes don't all register with the FE at the exact same millisecond. * Initial Load: As the FE's Tablet Scheduler comes online and sees the first few nodes become

Alive

, it may start assigning tablet "primary" responsibilities to them immediately to ensure data availability. * Metadata Stickiness: While StarRocks tries to maintain a balanced state, the initial recovery phase favors availability. If the first 3 nodes of a 10-node cluster register first, they may be assigned a heavy load before the remaining 7 nodes are even visible to the FE. 3. Why did recreating the table improve distribution? Recreating the table follows a different code path than cluster recovery: * Global Placement Logic: At table creation time, the FE performs a "Fresh Placement." It looks at the entire set of currently healthy nodes and uses a round-robin or load-aware algorithm to spread the new tablets evenly across all of them at once. * No Legacy State: Unlike a restart where the FE might be trying to reconcile old assignments or dealing with rolling node heartbeats, a

CREATE TABLE

starts with a blank slate and a full view of the cluster capacity, leading to a much more deterministic and balanced initial layout. Recommendation for v4.1.1: StarRocks 4.1 introduced Adaptive Layout (automatic bucket adjustment), which is designed to handle these skews dynamically. If your table is using a fixed bucket count, you might consider using Random Bucketing or ensuring

bucket_size

is set in properties, which allows the system more flexibility to balance the load as it grows. References • table_design/data_distribution/Data_distribution.md • administration/management/FE_configuration.md • faq/operation_maintenance_faq.md • sql-reference/sql-statements/table_bucket_part_index/ALTER_TABLE.md