<@U0A71G31CDV> I'm using 3.5.20.I added the iceber...
# questions-and-troubleshooting
m
@Rocky I'm using 3.5.20.I added the iceberg catalog, and when I query the iceberg tables through this catalog, I get the following error: "Fail to find valid type encoding, type:BYTE_ARRAY, encodingDELTA LENGTH BYTE ARRAY file = ...parquet: BE:10600" Is there any solution to this problem?
r
The error you are encountering—`Fail to find valid type encoding, type:BYTE_ARRAY, encoding:DELTA_LENGTH_BYTE_ARRAY`—is caused by querying Parquet files that use Parquet V2 encodings.
DELTA_LENGTH_BYTE_ARRAY
is a newer encoding used for string/byte array data to improve compression. As of StarRocks 3.5.20, the native C++ vectorized reader does not yet support this specific encoding. Support for this and other Parquet V2 delta encodings was implemented in later development cycles (via Pull Request #57954) and is primarily available in StarRocks 3.6 and newer versions (such as 4.0 and 4.1). Recommended Solutions 1. Upgrade to StarRocks 4.0 or newer: Support for Parquet V2 delta encodings (including
DELTA_LENGTH_BYTE_ARRAY
and
DELTA_BINARY_PACKED
) has been fully integrated into the native reader starting with the 4.0 release. If possible, upgrading is the most permanent and performance-efficient solution. 2. Use the JNI Scanner (Workaround): If you cannot upgrade immediately, you can try switching to the JNI-based scanner. The JNI scanner uses the Java Parquet library, which typically supports a wider range of encodings than the native vectorized reader.
Copy code
sql
    SET enable_jni_scanner = true;
    -- Or for some versions specifically:
    SET enable_iceberg_jni_scanner = true;
Note: This may result in slower query performance compared to the native reader. 3. Adjust Parquet Writer Settings: If you have control over the system writing the Iceberg tables (e.g., Spark, Flink, or Athena), you can configure it to use Parquet V1 or disable delta encoding for byte arrays. * In Spark, you can set
parquet.writer.version
to
v1
. * In Iceberg table properties, ensure
write.parquet.format-version
is set to
1
(or avoid explicitly enabling V2 features). Summary of Support * StarRocks 3.5.x and below: Limited support for Parquet V2 delta encodings in the native reader. * StarRocks 3.6 / 4.0+: Full Support:** Full native support for
DELTA_LENGTH_BYTE_ARRAY
,
DELTA_BYTE_ARRAY
, and
DELTA_BINARY_PACKED
. References • data_source/catalog/iceberg/iceberg_catalog.md • data_source/feature-support-data-lake-analytics.md