@Rocky hey rocky using external catalog with iceberg. reading varchar or varbinary type can be more 10mb?

When using a StarRocks External Catalog with Iceberg, the answer is generally no for standard

VARCHAR

and

VARBINARY

types. Here are the specific limits and details you should know: 1.

VARCHAR

and

VARBINARY

Limits In StarRocks, both

VARCHAR

and

VARBINARY

types have a maximum length of 1,048,576 bytes (1 MB). This is a system-level limit for these specific data types. Even when reading from external sources like Iceberg, StarRocks maps the Iceberg

string

or

binary

types to these internal types, meaning any single value exceeding 1 MB will typically result in an error or truncation during the scan. 2. Handling Data Larger Than 1 MB (Up to 16 MB) If you need to read individual values that are larger than 1 MB (like your 10 MB requirement), you can use the following types if your StarRocks version supports them: *

JSON

Type: Supported since version 2.2, the

JSON

type allows for a maximum length of 16 MB. If your data is JSON-formatted, mapping it to this type can bypass the 1 MB limit. *

VARIANT

Type: Available in StarRocks 3.2 and later, the

VARIANT

type also supports up to 16 MB. It is specifically designed to handle semi-structured data from external formats like Iceberg (Parquet) and is highly efficient for these scenarios. Summary Table Type | Max Size | Notes

VARCHAR

| 1 MB | Default mapping for Iceberg strings.

VARBINARY

| 1 MB | Default mapping for Iceberg binary.

JSON

| 16 MB | Use for JSON-formatted strings.

VARIANT

| 16 MB | Recommended for large/semi-structured data (v3.2+). Recommendation: If your Iceberg table contains strings or binary blobs larger than 1 MB, you won't be able to read them using the standard

VARCHAR

or

VARBINARY

columns. You should check if you can leverage the

VARIANT

type or ensure that the data is partitioned/structured such that individual values do not exceed the 1 MB threshold. References • knowledge/usage/best-practice.md • data_source/catalog/iceberg/iceberg_catalog.md • release_notes/release-2.1.md • release_notes/release-4.0.md