@Rocky, Can

Label Already Exists

error come for failed stream load transaction?

Yes, the

Label Already Exists

error can occur in the context of a failed or problematic stream load transaction, but usually because of the timing and state of that transaction. Here is how it happens for failed transactions: 1. The "Still Running" Scenario (Most Common) If a Stream Load request times out on the client side (e.g., your HTTP client cuts the connection), the StarRocks backend may still be processing that transaction. • If you immediately retry the load with the same label, StarRocks sees the first transaction is still in the

RUNNING

or

PREPARE

state. • In this case, StarRocks returns

Label Already Exists

because the label is currently "owned" by an active process, even though the client thinks it "failed" (timed out). 2. Transaction Interface Specifics If you are using the Stream Load Transaction Interface (

/api/transaction/begin

): • If you call

/begin

with a label that is already associated with an ongoing transaction, you will receive

Status: LABEL_ALREADY_EXISTS

with

ExistingJobStatus: RUNNING

. • Some versions of StarRocks documentation note that initiating a new transaction with an existing label will cause the previous ongoing transaction to be rolled back/aborted, but the new request still receives the error to prevent accidental overlaps. 3. When can you reuse a label? You can only reuse a label if the previous job is in the

CANCELLED

(failed/aborted) state. • FINISHED: You cannot reuse the label. You will always get

Label Already Exists

. • RUNNING / PREPARE: You cannot reuse the label yet. You will get

Label Already Exists

. • CANCELLED / ABORTED: You can reuse the label. StarRocks allows retrying a failed job using the same label to ensure Exactly-Once semantics (the second attempt will succeed if the first one truly failed and stopped). Summary of causes for this error during "failures": * Automatic Retries: Your HTTP client (like

curl

or a Python library) automatically retried a request that it thought failed, but the backend received both. * Lingering Transactions: A previous attempt crashed or was interrupted, and StarRocks hasn't yet moved it to the

CANCELLED

state (it might be waiting for a timeout). * Label Conflict: You accidentally used a label that was already successfully used by another job in the past (within the retention period, typically 3 days by default). Recommendation: If you see this error, run

SHOW LOAD WHERE LABEL = "your_label";

to check the actual state of the job. • If it is

FINISHED

, you must use a new label. • If it is

RUNNING

, wait for it to finish or fail before retrying, or increase your client-side timeout. References • faq/loading/Loading_faq.md • loading/loading_introduction/loading_concepts.md • administration/management/resource_management/be_label.md • faq/loading/Broker_load_faq.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • loading/Stream_Load_transaction_interface.md • loading/Flink-connector-starrocks.md

@Rocky, What is best practice is whether to generate new label for each retry (for retryable http errors only like timeout) for a batch into primary key type table with shared-nothing architecture?

The best practice for StarRocks Stream Load (including Primary Key tables and shared-nothing architecture) is to reuse the same label for retries of the same batch of data. Generating a new label for every retry is discouraged because it breaks StarRocks' built-in exactly-once guarantee. Why you should reuse the same label: 1. Exactly-Once Guarantee: StarRocks uses the label to ensure idempotency. If a retryable error (like a client-side timeout) occurs, the server might have actually received and successfully committed the data. Reusing the label prevents the same data from being loaded twice. 2. Handling "Label Already Exists": This error is actually a "safety feature." If you get this error during a retry: * It means the first attempt might have succeeded or is still in progress. * You should check the status using

SHOW LOAD WHERE LABEL = 'your_label';

. If it is

FINISHED

, your job was successful and you can move to the next batch. 3. Efficiency: Reusing the label and letting StarRocks handle the "already exists" state is more efficient than generating a new label and forcing the database to perform another full write/upsert of the same data. Specific Considerations for Primary Key (PK) Tables: While PK tables are "upsert-based" (meaning double-loading the same data usually results in the same final state), reusing the same label is still superior for several reasons: * Write Amplification: Retrying with a new label causes StarRocks to process the same data again, consuming CPU, IO, and memory for redundant work. * Version History: In PK tables, every write creates a new version. Redundant writes increase the pressure on the background compaction process. * Consistency (Out-of-order writes): If you use new labels for retries, you risk a race condition where an older retry finishes after a newer batch has already started loading. While PK tables have mechanisms like Conditional Update (using a sequence/timestamp column) to mitigate this, using the same label for the same batch is a cleaner way to ensure that "Batch A" is treated as a single atomic unit of work regardless of how many network retries it takes. Summary of the Best Practice Workflow: 1. Generate a unique label for each new batch of data (e.g.,

batch_20231027_001

). 2. Submit the Stream Load. 3. On Timeout / Retryable Error: Retry the exact same request with the same label. 4. On `LABEL_ALREADY_EXISTS`: * Call

SHOW LOAD WHERE LABEL = 'your_label';

. * If

State

is `FINISHED`: Success. * If

State

is `CANCELLED`: Retry again (same label). * If

State

is `RUNNING`: Wait and check again. 5. Adjust Timeouts: If you frequently see

LABEL_ALREADY_EXISTS

due to timeouts, increase your client-side HTTP timeout to match your data volume. References • faq/loading/Loading_faq.md • loading/loading_introduction/loading_concepts.md • sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • loading/Flink-connector-starrocks.md • best_practices/primarykey_table.md References • faq/loading/Loading_faq.md • loading/loading_introduction/loading_concepts.md • faq/loading/Broker_load_faq.md