Hi, everyone teacher. I would like to ask a question about a routine load task. Currently, there are messages containing non-JSON prefixes or suffixes. It first entered the PAUSED state. After modifying the error count, it is now stuck in the RUNNING state. Correct data has been provided, but the data cannot be written into the table. Experiment steps: Routine load design: ( "max_batch_rows" = "200000", "desired_concurrent_number" = "2", "jsonpaths" = "[\"$.uid\", \"$.name\", \"$.age\"]", "format" = "json", "max_error_number" = "0", "max_filter_ratio" = "0" ) Operation steps: 1. Send a correct data message: {"uid": 1001, "name": "张三", "age": 25} → Writes successfully. 2. Send an incorrect data message: {"uid": 1011, "name": "李十三", "age":} → Cannot write, state changes to PAUSED. 3. Execute: ALTER ROUTINE LOAD FOR tmp.rload__ods_kfk__test_di_topic_test2 PROPERTIES ( "max_error_number" = "3", -- Allows 10% error data "max_filter_ratio" = "1" ); 4. Send a correct data message: {"uid": 1016, "name": "黄十八", "age": 27} → Writes successfully. 5. Send an incorrect data message: {"uid": 1011, "name": "李十三", "age":} → Cannot write, state remains RUNNING. 6. Send a correct data message: {"uid": 1017, "name": "赵十九", "age": 30} → State still RUNNING, data cannot be written, stuck.

Hello, @yuhui zhang! How can I assist you today?

Hi, everyone teacher. I would like to ask a question about a routine load task. Currently, there are messages containing non-JSON prefixes or suffixes. It first entered the PAUSED state. After modifying the error count, it is now stuck in the RUNNING state. Correct data has been provided, but the data cannot be written into the table. Experiment steps: Routine load design: ( "max_batch_rows" = "200000", "desired_concurrent_number" = "2", "jsonpaths" = "[\"$.uid\", \"$.name\", \"$.age\"]", "format" = "json", "max_error_number" = "0", "max_filter_ratio" = "0" ) Operation steps: 1. Send a correct data message: {"uid": 1001, "name": "张三", "age": 25} → Writes successfully. 2. Send an incorrect data message: {"uid": 1011, "name": "李十三", "age":} → Cannot write, state changes to PAUSED. 3. Execute: ALTER ROUTINE LOAD FOR tmp.rload__ods_kfk__test_di_topic_test2 PROPERTIES ( "max_error_number" = "3", -- Allows 10% error data "max_filter_ratio" = "1" ); 4. Send a correct data message: {"uid": 1016, "name": "黄十八", "age": 27} → Writes successfully. 5. Send an incorrect data message: {"uid": 1011, "name": "李十三", "age":} → Cannot write, state remains RUNNING. 6. Send a correct data message: {"uid": 1017, "name": "赵十九", "age": 30} → State still RUNNING, data cannot be written, stuck. @Rocky

Hello! I am Rocky, the StarRocks assistant. I can help you troubleshoot why your Routine Load task appears "stuck" in the

RUNNING

state without loading new data. Based on your description, the job is likely encountering one of the following scenarios: 1. Cumulative Error Count Reaching the Limit In StarRocks,

max_error_number

is a cumulative count for the job's lifetime. When you

ALTER

the job to increase the limit, the previous error counts are not reset. * If your job had errors previously (from Step 2 and Step 5), and the total count reaches 3, the job will transition to

PAUSED

. * However, if it is still

RUNNING

, it means the limit hasn't been hit yet, or the task is still attempting to process. 2. Batch Interval and Commit Delay Routine Load does not write data to the table for every single message. It commits data in batches based on: *

max_batch_rows

(default 200,000) *

max_batch_interval

(default 10 seconds) If you only send 1 or 2 messages, they will not appear in the table until the

max_batch_interval

is reached. Since your

max_batch_rows

is very large (200,000), it is almost certainly waiting for the time interval to expire before committing. 3. Data is being Filtered (max_filter_ratio = 1) Since you set

"max_filter_ratio" = "1"

, StarRocks will allow 100% of the data to fail without pausing the job. * If the "correct" data in Step 6 actually has a slight mismatch with your

jsonpaths

or the table schema, it will be filtered out. * The job will stay

RUNNING

, but no data will be written to the table. ────────── Troubleshooting Steps To find the exact cause, please execute the following command and examine the output:

sql
SHOW ROUTINE LOAD FOR tmp.rload__ods_kfk__test_di_topic_test2\G

Please check these specific fields in the result: 1.

Statistic

: * Compare

loadedRows

and

errorRows

. If

errorRows

is increasing but

loadedRows

is not, your "correct" data is actually being rejected. 2.

Progress

vs.

LatestSourcePosition

: *

Progress

shows the offset StarRocks has successfully processed. *

LatestSourcePosition

shows the latest offset available in Kafka. * If

Progress

matches

LatestSourcePosition

, StarRocks has consumed the message. If it's still not in the table, it was either filtered or the transaction hasn't committed yet. 3.

ErrorLogUrls

: * If

errorRows

is greater than 0, copy the URL provided in this field and open it in your browser (or use

curl

). It will show the exact reason why the rows were rejected (e.g., "JSON parse error" or "column mismatch"). 4.

ReasonOfStateChanged

: * Even if the state is

RUNNING

, this field might contain information about the last reason the job was paused or encountered an issue. Recommendation If you want to see data faster for testing, you can temporarily reduce the `max_batch_interval`:

sql
ALTER ROUTINE LOAD FOR tmp.rload__ods_kfk__test_di_topic_test2
PROPERTIES (
    "max_batch_interval" = "5" -- Commit every 5 seconds
);

Also, ensure that your Kafka message does not contain the "non-JSON prefixes or suffixes" you mentioned, as the

format="json"

setting requires the entire Kafka message to be a valid JSON object. If there is extra text, the parser will fail and the row will be counted as an error. References • faq/loading/Routine_load_faq.md • loading/RoutineLoad.md • sql-reference/sql-statements/loading_unloading/routine_load/SHOW_ROUTINE_LOAD.md

@Rocky useless ，I don't know bug

I'm sorry to hear you're having a frustrating time! I'd really like to help you figure out what's going on. To get to the bottom of the "bug" or issue, could you share a bit more detail? For example: 1. What are you trying to do? (e.g., starting the cluster, running a specific SQL query, or loading data). 2. What error message are you seeing? If you have a snippet of the error from your terminal or SQL client, that would be very helpful. 3. Check the logs: Often the most useful information is hidden in the logs. You can check these files for "WARN" or "ERROR" messages: * FE (Frontend):

fe/log/fe.warn.log

* BE (Backend):

be/log/be.WARNING

If you can provide the SQL query or the specific error text, I can search the documentation for a solution for you! References • deployment/deploy_manually.md • faq/Sql_faq.md • using_starrocks/async_mv/troubleshooting_asynchronous_materialized_views.md • developers/build-starrocks/Build_in_docker.md • faq/Deploy_faq.md • faq/Others.md • developers/mac-compile-run-test.md • administration/management/monitoring/alert.md