@Rocky I have error "[42000][1064] Dict Decode failed, Dict can't take cover all key 0 BE:11001" while trying to parse json columnd table (which is also continous streaming table)

This error,

[42000][1064] Dict Decode failed, Dict can't take cover all key :0: BE:11001

, is a known issue in StarRocks related to Low-Cardinality Optimization (Global Dictionary). It occurs when the query engine expects a dictionary-encoded value to exist in its internal global dictionary, but the value is missing or the dictionary has become inconsistent. In the context of JSON columns and streaming tables, this is frequently triggered by the "Flat JSON" optimization (introduced in v3.3.0) which automatically extracts and encodes JSON sub-fields to improve performance. Recommended Solutions 1. Disable Low-Cardinality Optimization (Immediate Workaround) The most common and effective workaround is to disable the optimization that is failing. You can do this at the session level for your queries:

sql
-- Disable the general low-cardinality optimization
SET cbo_enable_low_cardinality_optimize = false;

-- If you are using StarRocks 3.3 or later, also try disabling JSON-specific dictionary optimization
SET cbo_json_v2_dict_opt = false;

If the error occurs during a background process (like an

INSERT INTO ... SELECT

within a Routine Load or Pipe), you may need to set these variables globally:

sql
SET GLOBAL cbo_enable_low_cardinality_optimize = false;

2. Disable JSON Path Rewriting If the error persists after disabling dictionary optimization, try disabling the JSON v2 rewrite feature, which stops the engine from trying to optimize JSON sub-path access via the new mechanism:

sql
SET cbo_json_v2_rewrite = false;

3. Handle Schema/Data Inconsistencies This error can sometimes be triggered by: * Schema changes: If you recently added columns or changed types. * Special Characters: Values that the dictionary cannot handle correctly (e.g., certain multi-byte or traditional Chinese characters). * Stale Metadata: For Primary Key tables, background compactions or updates can occasionally lead to dictionary inconsistencies. 4. Permanent Fix: Upgrade StarRocks This issue is a known bug in several versions. Fixes were implemented in the following versions: * v2.5.x: Fixed in 2.5.10+ * v3.0.x: Fixed in 3.0.4+ * v3.1.x and above: Ensure you are on the latest patch release (e.g., 3.1.10+, 3.2.5+). If you are on a version earlier than these, upgrading to the latest maintenance release of your major version is the recommended permanent solution. Troubleshooting Tip If the error only happens on a specific table, you can try to "refresh" the internal state by: 1. Running

ANALYZE TABLE <table_name>;

to update statistics. 2. If it's a small table, a

TRUNCATE

and reload may resolve the dictionary inconsistency. References • administration/management/BE_configuration.md • using_starrocks/Flat_json.md

@Rocky so why there can be a problem with the continious streaming injestion? time to time we can seleect

The intermittent nature of this error during continuous streaming is typically due to a metadata race condition between your ingestion process and the query engine. Here is why it happens "from time to time": 1. The Global Dictionary Race Condition When you query a table with JSON or String columns, StarRocks often uses Low-Cardinality Optimization. This involves building a "Global Dictionary" that maps strings to integers to speed up the query. * During Query Planning: The Frontend (FE) checks the current version of the data and the dictionary metadata. * During Continuous Ingestion: Your streaming job (Routine Load or Stream Load) is constantly committing new "versions" of data (rowsets). * The Conflict: If a new batch of data is committed or a background compaction finishes between the time the FE plans your query and the Backend (BE) actually reads the data, the BE may encounter data that the FE's dictionary doesn't yet "cover." This leads to the

Dict can't take cover all key

error. 2. Flat JSON Optimization (v3.3+) If you are on StarRocks 3.3 or higher, the Flat JSON feature automatically extracts sub-fields from your JSON. To make these sub-fields performant, StarRocks automatically applies dictionary encoding to them. In a high-velocity streaming environment, these sub-fields are constantly evolving, increasing the likelihood that the dictionary state becomes temporarily inconsistent. 3. Small File / Compaction Pressure Continuous streaming often creates many small files. To maintain performance, StarRocks runs background compaction to merge these files. Compaction frequently updates the internal mapping of data. If the query hits a tablet exactly while its version is being updated or compacted, the dictionary decoding can fail. How to stabilize your streaming table To stop the intermittent errors without stopping your ingestion, you can apply these settings: A. Disable the specific optimization (Recommended for JSON) This tells the engine to stop relying on the potentially stale dictionary and instead read the raw values. It is the most common fix for streaming JSON issues.

sql
-- Disable for the current session to test
SET cbo_enable_low_cardinality_optimize = false;
SET cbo_json_v2_dict_opt = false; -- Specifically for JSON sub-fields

B. Increase Ingestion Batch Size If your streaming job commits every 1 second, it creates high pressure. Try to increase the

desired_max_interval

in Routine Load (e.g., to 5 or 10 seconds) or increase the batch size. Fewer commits lead to fewer version changes and a more stable global dictionary. C. Update Statistics Sometimes the FE has outdated information about the column cardinality. Running a manual analysis can help:

sql
ANALYZE TABLE <your_table_name>;

Summary: The error is not a sign of data corruption; it is a "mismatch" between the query's plan and the rapidly changing state of your streaming data. Disabling

cbo_enable_low_cardinality_optimize

is the standard way to trade a small amount of performance for 100% query stability in query stability. References • administration/management/BE_configuration.md • using_starrocks/distinct_values/Using_bitmap.md • administration/Meta_recovery.md

@Rocky that SET cbo_json_v2_dict_opt = false helped so what can be a problem? and increasing streaming time can help?

Since

SET cbo_json_v2_dict_opt = false

solved the problem, it confirms that the error was caused by stale or inconsistent Global Dictionary metadata specifically for your flattened JSON sub-columns. What exactly is the problem? When Flat JSON is enabled (automatic in v3.3+), StarRocks extracts common fields from your JSON and stores them as internal sub-columns. To speed up queries, it builds a Global Dictionary for any string-based sub-fields. The error happens because of a race condition between your ingestion and your queries: 1. The Planner Phase: When you run a query, the Frontend (FE) looks at the current data version and prepares a dictionary map for the "low-cardinality" values it expects to see. 2. The Execution Phase: Because you are streaming data continuously, a new batch of data is committed or a background compaction finishes after the query was planned but before the Backend (BE) finished reading. 3. The Mismatch: The BE encounters a new value (or a new version of the data) that wasn't in the dictionary the FE provided at the start of the query. The BE then fails with

Dict can't take cover all key

because it doesn't know how to decode that value. Why does increasing streaming time (batching) help? Yes, increasing the ingestion interval (e.g., from 1 second to 10 seconds) will significantly help stabilize the table. Here is why: * Fewer Metadata Versions: Every time a streaming load commits, it creates a new "version" of the table metadata. If you commit every second, the FE and BE have to sync thousands of versions an hour. Larger batches mean fewer versions and a more stable state for the query engine. * Reduced Compaction Pressure: Frequent small imports trigger aggressive background "compaction" (merging files). Compaction is the most common time for dictionary/version mismatches to occur. By batching more data, you create fewer small files, which reduces the frequency of these background updates. * Dictionary Stability: The Global Dictionary is updated during the load process. If you load data less frequently, the dictionary remains "static" for longer periods, giving your queries a consistent window to execute without the version changing mid-stream. Recommendation If you want to keep the performance benefits of Flat JSON without the errors: 1. Increase `desired_max_interval`: If using Routine Load, set this to at least 5–10 seconds. 2. Increase Batch Size: Ensure your loads are at least 100MB+ or contain several hundred thousand rows per commit if possible. 3. Keep the Workaround: If your data has very high cardinality (too many unique strings in the JSON fields), the dictionary optimization won't help much anyway. Leaving

cbo_json_v2_dict_opt = false

is a perfectly valid long-term solution for high-velocity streaming tables. References • using_starrocks/Flat_json.md

@Rocky may be when json has lists in it thats can be a problem?

Yes, lists (arrays) inside your JSON can definitely be the cause, especially when combined with continuous streaming. There are three specific reasons why JSON arrays trigger this "Dict Decode" error: 1. The "Type Inconsistency" Problem The Flat JSON feature works by "guessing" the schema of your JSON based on the first few thousand rows. * If a field (e.g.,

user_tags

) is a String in some rows but a List/Array in others, the Global Dictionary can become corrupted. * The query planner might create a dictionary expecting simple strings, but when the streaming ingestion brings in a row where that same path contains an array, the dictionary decoder fails because it doesn't know how to "map" an array value into a string dictionary. 2. Flat JSON Limitation with Arrays According to the StarRocks documentation, Flat JSON cannot materialize keys located inside JSON Arrays. It only materializes common keys in JSON Objects. * If you have

{"data": {"name": "Rocky"}}

, the

name

field can be flattened and dictionary-encoded. * If you have

{"data": [{"name": "Rocky"}]}

, the

name

field cannot be flattened. * If your data structure switches between these two formats frequently during streaming, the engine's attempt to optimize that path (Low-Cardinality Optimization) will fail intermittently as the "sub-column" disappears or changes type. 3. High Cardinality in Lists If you are querying strings inside an array (e.g.,

WHERE array_contains(cast(json_col->'tags' as array<string>), 'blue')

), and those strings have a very high number of unique values, the Global Dictionary will frequently reach its limit or become "stale" as new unique values arrive via streaming. Does increasing streaming time help? Yes. Increasing the interval (e.g., from 1s to 10s) helps because: * Version Stability: Every commit creates a new version of the table. A query running during a "commit storm" is likely to see a data version that is newer than the dictionary metadata it started with. * Compaction Frequency: Fewer commits mean the background process that "merges" these JSON schemas (compaction) runs less often, giving the Global Dictionary a more stable environment. Final Recommendation Since

cbo_json_v2_dict_opt = false

fixed the error, the most stable setup for a JSON streaming table with nested lists is: 1. Keep

cbo_json_v2_dict_opt = false

for that table or session. The performance loss is usually negligible for complex/nested JSON, and it prevents 100% of these dictionary errors. 2. Increase Ingestion Interval: Set your Routine Load

desired_max_interval

to at least 5–10 seconds. 3. Check for Type Consistency: If possible, ensure that a specific JSON path always contains the same type (e.g., always a String OR always a List, but not both). References • using_starrocks/Flat_json.md • sql-reference/data-types/semi_structured/JSON.md • administration/management/BE_configuration.md References • faq/loading/Routine_load_faq.md • administration/http_interface/query_detail.md • sql-reference/data-types/semi_structured/JSON.md • loading/Json_loading.md • sql-reference/sql-functions/json-functions/json-constructor-functions/json_array.md