@Rocky, I've ended up with some poorly distributed data on a table after a few large inserts. The average compaction score is down to double digits but the MaxCS is 500. I'm assuming this implies there's one oversized file:

mysql> SHOW PARTITIONS FROM data_lake.pipeline_generation;
*************************** 1. row ***************************
      PartitionId: 2064229
    PartitionName: pipeline_generation
   CompactVersion: 10722
   VisibleVersion: 10722
      NextVersion: 10723
            State: NORMAL
     PartitionKey:
            Range:
  DistributionKey: master_uuid
          Buckets: 72
         DataSize: 4.3GB
      StorageSize: 4.3GB
         RowCount: 80568218
  EnableDataCache: true
       AsyncWrite: false
            AvgCS: 19.29
            P50CS: 0.00
            MaxCS: 500.00
      DataVersion: 10439
     VersionEpoch: 424854433556332544
   VersionTxnType: TXN_NORMAL
MetaSwitchVersion: 0

I've run

ALTER TABLE pipeline_generation COMPACT;

a few times and the AvgCS keeps going down but I can't get the MaxCS to go down. Is there a way I can force it? Is my assumption about an oversized file correct or is something else going on here? Is this even a problem or am I overthinking it?

Your assumption that a high MaxCS (Max Compaction Score) with a low AvgCS implies a specific bottleneck is correct, but it likely indicates data skew across your buckets rather than a single "oversized file." In StarRocks shared-data (lake) clusters, a MaxCS of 500.00 is a specific threshold. According to the documentation, a score over 100 is unhealthy, and exceeding 500 indicates a very high score where manual intervention is often required. What does MaxCS 500 mean? * Tablet-level Bottleneck: Your table has 72 buckets (tablets).

AvgCS

is 19.29, which is relatively healthy for the table as a whole. However,

MaxCS: 500

means at least one specific tablet has 500 unmerged rowsets (versions) in its most crowded size tier. * Data Skew: Since the average is low but the max is at the warning ceiling, your data is likely skewed toward one or a few buckets. The distribution key

master_uuid

usually prevents this, but if there was a "hot key" or a specific insert pattern, one bucket may have received thousands of small files while others remained clean. * Compaction Limits: The fact that it is stuck exactly at 500 might be related to the parameter

max_cumulative_compaction_num_singleton_deltas

, which often defaults to 500. The system may be struggling to "bite off" enough files in a single cumulative compaction task to lower the score. How to Force It Down If

ALTER TABLE ... COMPACT

(which defaults to cumulative compaction) isn't moving the needle, you should try a Base Compaction. This forces the system to merge all existing rowsets into a single base version, which is the most effective way to "reset" a high score. Run this command:

sql
ALTER TABLE data_lake.pipeline_generation BASE COMPACT;

Troubleshooting Steps If the score still doesn't budge after a Base Compaction: 1. Check for Stuck Tasks: Run

SHOW PROC '/compactions';

to see if any compaction tasks are currently running or failing. If

CommitTime

or

FinishTime

is NULL for a long time, the task might be stuck. 2. Inspect Tablet Distribution: To confirm if skew is the issue, you can check the version count of individual tablets:

sql
    SHOW TABLET FROM data_lake.pipeline_generation;

Look for tablets with a significantly higher

Version

count or

DataSize

than others. 3. Check for Errors: If you are on a Primary Key table, check the detailed status of cloud-native compactions:

sql
    SELECT * FROM information_schema.be_cloud_native_compactions
    WHERE TABLE_NAME = 'pipeline_generation'
    ORDER BY START_TIME DESC;

Is this a problem? Yes, it can be. While an

AvgCS

of 19 won't kill cluster performance, a

MaxCS

of 500 on a specific bucket means that any query hitting that bucket has to open and merge 500 different file versions on the fly. This will cause tail latency spikes for your queries. If it reaches 2000, the system will begin rejecting new imports to that partition entirely. Recommendation: Run the

BASE COMPACT

command and monitor

SHOW PROC '/compactions'

to ensure it completes successfully. References • administration/management/compaction.md • administration/management/FE_configuration.md • knowledge/admin_and_operation/shared_data_compaction.md • administration/management/BE_configuration.md