@Rocky Can you please explain this error message from routine load?

[2026-05-14 20:29:41] [task id: 019e282e-44a3-7701-8459-0421e961eb7c] [txn id: -1] previous task aborted because of Failed to load data into tablet 264373, because of too many versions, current/limit: 1050/1000. You can reduce the loading job concurrency, or increase loading data batch size. If you are loading data with Routine Load, you can increase FE configs routine_load_task_consume_second and max_routine_load_batch_size,:  be:10.31.162.73

This error occurs because the data ingestion rate is higher than the rate at which StarRocks can merge (compact) the data. Why this happens Every time a Routine Load task commits, it creates a new version (a set of data files) for the affected tablets. StarRocks has a safety limit (default 1000) for the number of versions a tablet can have. If this limit is reached, the system stops accepting new data for that tablet to prevent a massive drop in query performance (since more versions mean more files to scan during a read). Recommended Solutions 1. Reduce commit frequency (Routine Load settings) The most effective way to fix this for Routine Load is to make each task run longer and carry more data, which reduces the number of versions created over time. You can modify your Routine Load job or update FE global configurations: *

routine_load_task_consume_second

: Increase this (e.g., from the default 15s to 60s or more). This forces tasks to wait longer before committing. *

max_routine_load_batch_size

: Increase this (default is ~4GB) to ensure tasks don't commit early due to size limits. *

desired_concurrent_number

: If you have many partitions in Kafka, reduce the concurrency of the load job so fewer tasks are committing versions simultaneously. 2. Speed up Compaction (BE settings) If your data volume is naturally high and you need frequent commits, you can tune the Backend (BE) to merge files faster. * For Primary Key tables: * Increase

update_compaction_num_threads_per_disk

(default 1). * Decrease

update_compaction_per_tablet_min_interval_seconds

(default 120) to allow compaction to run more frequently on the same tablet. * For Duplicate/Unique/Aggregate tables: * Increase

cumulative_compaction_num_threads_per_disk

. * Decrease

cumulative_compaction_check_interval_seconds

. 3. Increase the limit (Temporary fix) You can increase the version limit in

be.conf

, though this may slow down queries: *

tablet_max_versions

: Increase this to

2000

or higher. *

tablet_max_pending_versions

: (Specific to Primary Key tables) Increase if the error mentions pending versions. How to check compaction progress To see if compaction is struggling, you can run:

sql
SHOW PROC '/compactions';

Look for tablets with a high Score. A score consistently above 100 indicates that compaction cannot keep up with the ingestion rate. References • administration/management/BE_configuration.md • best_practices/primarykey_table.md • faq/loading/Loading_faq.md • knowledge/admin_and_operation/shared_data_compaction.md • administration/management/compaction.md