@Rocky Since we moved to stream load we are getting lot of errors related to

too many versions

and eventually failing with

quorum: 1<2

failure. We already have

tablet_max_version

raised to 3000, 11k Stream Load requests in 15 minutes, ~180 transaactions per min - batches of size 110-350.

The errors you're seeing are directly caused by the high frequency of your Stream Load requests. With 11,000 requests in 15 minutes (~12 requests per second), you are creating a massive number of data versions (rowsets) faster than the system can merge them (compaction). When multiple backends (BEs) hit the

tablet_max_version

limit or become overloaded by the version metadata, they begin rejecting writes. The

quorum: 1<2

error is a symptom of this: it means the write failed because it couldn't be committed to a majority of replicas (e.g., only 1 replica succeeded out of 3, but 2 were required). Recommended Solutions 1. Batch Your Data (Most Effective) StarRocks performs best when you load larger chunks of data less frequently. * Current: ~733 loads/min with 110–350 rows. * Target: Aim for one load every 5–10 seconds per table. * Action: Buffer your data on the client side (e.g., in your application or a queue like Kafka) and send batches of 10MB–100MB or at least tens of thousands of rows. 2. Enable "Merge Commit" (If on v3.4.0+) If you cannot easily change your client-side batching logic, StarRocks v3.4.0 introduced Merge Commit. This feature automatically merges multiple concurrent Stream Load requests into a single transaction on the server side. * How to use: Set the following header in your Stream Load request:

"merge_commit: true"

* This significantly reduces the number of versions generated without requiring you to change your upstream application logic. 3. Accelerate Compaction (BE Config) You can make the background compaction process more aggressive to keep up with the version creation. In your

be.conf

, adjust the following (and restart BEs): * For Shared-nothing (standard) clusters:

properties
    # Increase threads for merging small versions
    cumulative_compaction_num_threads_per_disk = 4
    # Check for compaction tasks more frequently
    cumulative_compaction_check_interval_seconds = 2

* For Shared-data (compute-storage separation) clusters:

sql
    -- Run via MySQL client
    UPDATE information_schema.be_configs SET value = "8" WHERE name = "compact_threads";
    -- Increase version publishing speed on FE
    ADMIN SET FRONTEND CONFIG ("lake_enable_batch_publish_version" = "true");

4. Use the Transaction Interface If you are loading from a custom application, use the Stream Load Transaction Interface. This allows you to open a transaction, send multiple small HTTP "mini-batches," and then

COMMIT

once to generate only one version for the entire set. Summary of what to check next: 1. Check Compaction Status: Run

SHOW PROC '/compactions';

to see the

max_compaction_score

. If it's consistently high (e.g., > 100), compaction is lagging. 2. Verify Replica Health: Use

SHOW TABLET <tablet_id>;

on a failing tablet to see if specific replicas are stuck at a high version count. References • faq/loading/Stream_load_faq.md • loading/StreamLoad.md • administration/management/monitoring/metrics.md • loading/Stream_Load_transaction_interface.md • faq/shared_data_faq.md • knowledge/usage/load/shared_data_batch_ingestion.md