StarRocks does not have a native "Dead Letter Queue" (DLQ) feature that automatically redirects failed Kafka messages to another topic or a specific user-defined table in real-time. However, StarRocks provides several mechanisms to capture, log, and query failed messages for troubleshooting. 1. Accessing Failed Messages via URL By default, Routine Load captures records that fail due to data quality issues (e.g., parsing errors, data type mismatches). *

ErrorLogUrls

: When you run

SHOW ROUTINE LOAD FOR <job_name>\G

, you will see a field called

ErrorLogUrls

. This provides a URL (hosted on the BE nodes) where you can view or download the specific rows that failed and the reason for the failure. * Retention: By default, these logs are kept for 48 hours (controlled by the BE config

load_error_log_reserve_hours

). 2. Querying Failed Records with SQL (v3.1+) Starting from StarRocks v3.1, you can query failed records directly using system views: *

information_schema.loads

: This view contains a

TRACKING_SQL

field. If a load has failed rows, this field provides a pre-constructed SQL query that you can run to view the error logs. *

information_schema.load_tracking_logs

: You can query this view directly by filtering on the

LABEL

or

ID

of your load job to see the

TRACKING_LOG

(the specific data and error reason). *

log_rejected_record_num

: In your

CREATE ROUTINE LOAD

properties, set this to

-1

to log all failed records, or a specific positive integer

n

to log up to

n

records. 3. Error Tolerance Settings You can control how many failed messages are allowed before the job pauses: *

max_error_number

: The maximum number of error rows allowed before the job pauses (default is

0

). *

max_filter_ratio

: The percentage of failed rows allowed (0.0 to 1.0). If you want the job to keep running regardless of errors, you can set this to

1.0

. Common Workarounds for a "DLQ Table" If you strictly need failed messages in a StarRocks table for automated processing: 1. The "Varchar Staging" Strategy: Define a staging table where every column is a `VARCHAR`/`STRING`. Load everything from Kafka into this table (which prevents parsing errors). Then, use a SQL

INSERT INTO ... SELECT ...

with transformation logic to move valid data to your final table. You can then identify "failed" rows in the staging table using SQL filters. 2. External Scripting: Use a small script or cron job to fetch the data from the

ErrorLogUrls

or

load_tracking_logs

and insert it into a dedicated "dead letter table." 3. Upstream Processing: Handle DLQ logic before the data reaches StarRocks using a tool like Flink or a Kafka Streams application, which can branch invalid data to a separate "error" topic. References • sql-reference/sql-statements/loading_unloading/routine_load/CREATE_ROUTINE_LOAD.md • sql-reference/information_schema/load_tracking_logs.md • loading/RoutineLoad.md • loading/loading_introduction/troubleshooting_loading.md References • sql-reference/information_schema/loads.md • loading/loading_introduction/troubleshooting_loading.md • sql-reference/information_schema/load_tracking_logs.md