@Rocky What is diffence between string and varchar? would you recommend using

STRING

for regular non-key MySQL CDC columns in StarRocks? MySQL

VARCHAR(n)

is character-based, while StarRocks

VARCHAR(n)

is byte-based. We have already had strict-mode load failures when Korean and other multibyte values exceeded the StarRocks byte limit. We could use

VARCHAR(n * 4)

, but maintaining that conversion and fixing missed cases requires expensive schema changes. One affected table has 144 million rows and roughly 31,000 tablets. We are considering: - Non-key MySQL

VARCHAR

value columns → StarRocks

STRING

- PK, partition, bucketing, sort-key and internal identifier columns → bounded

VARCHAR

Would

STRING

introduce meaningful memory, optimizer, compaction, or query-performance costs in StarRocks 4.0 shared-nothing Primary Key tables, or is this a reasonable operational trade-off?

In StarRocks, the primary difference between

STRING

and

VARCHAR(n)

relates to schema constraints and memory management rather than storage efficiency. 1. Difference between

STRING

and

VARCHAR

Feature |

VARCHAR(n)

|

STRING

Length Definition | Required (unit: bytes). | Not required (uses a default max). Max Length | Up to 1,048,576 bytes (since v2.1). | 65,533 bytes (in most versions). Storage Cost | Same (actual data length). | Same (actual data length). Memory Cost | Based on actual length for most operations, but some query steps pre-allocate based on

n

. | Pre-allocates memory based on its maximum length (65,533). Index Usage | Can be part of Sort Key/Prefix Index (truncated). | Cannot be part of a Sort Key or Prefix Index. 2. Is it recommended for MySQL CDC? Yes, it is a recommended operational trade-off for non-key columns. Using

STRING

(or a very large

VARCHAR

) for non-key columns avoids the frequent

strict_mode

load failures caused by the character-vs-byte discrepancy. Since StarRocks

VARCHAR

is byte-based and MySQL's is character-based, a

VARCHAR(255)

in MySQL can hold up to 1,020 bytes in UTF-8 (4 bytes per char), which would fail in StarRocks if defined as

VARCHAR(255)

. 3. Costs and Performance Implications For StarRocks 4.0 Shared-nothing Primary Key tables, here is the impact of using `STRING`: * Memory (Query Execution): This is the most significant cost. During complex queries (like joins or aggregations), StarRocks may pre-allocate memory buffers based on the column's defined maximum length. Using

STRING

for dozens of columns can lead to much higher memory consumption per query, potentially causing OOMs or reducing the number of concurrent queries the cluster can handle. * Optimizer: There is no meaningful difference in the optimizer's ability to plan queries for non-key columns. However,

STRING

columns do not benefit from the 36-byte Prefix Index (sparse index). Since you are only using them for non-key columns, this is acceptable. * Compaction & Storage: There is no cost here. Both types use the same underlying columnar storage (encoding and compression). Compaction is triggered by version counts and data size, which remain the same regardless of whether you use

STRING

or a bounded

VARCHAR

. * Primary Key Model Specifics: * Delete Vectors: The cost of managing delete vectors and the Primary Key Index depends on the Primary Key columns, not the value columns. Your plan to keep PKs as bounded

VARCHAR

is correct and avoids increasing the memory footprint of the

Persistent Index

. * Partial Updates: Using

STRING

does not hinder partial column update performance. Recommendation Your strategy is sound for balancing operational stability with performance: 1. Keys & Structural Columns: Use bounded

VARCHAR(n)

. Ensure

n

is calculated as

MySQL_Chars * 4

to prevent ingestion errors. 2. Non-Key Value Columns: Use

STRING

. This removes the maintenance burden of schema changes for 144M+ row tables. 3. Refinement: If you have columns you know will never exceed a certain size (e.g., a "Status" code), prefer

VARCHAR

to save execution memory. If a column might exceed 65,533 bytes, you must use

VARCHAR(1048576)

instead of

STRING

. Warning: Do not use

STRING

for columns that will be frequently used in

GROUP BY

or

JOIN

conditions if performance is critical, as the memory pre-allocation for these operations is where the

STRING

type's "hidden cost" lies. References • faq/table_design_faq.md • knowledge/usage/best-practice.md • faq/Others.md • sql-reference/data-types/string-type/VARCHAR.md