Hi team, I’m using the Spark-StarRocks connector to write a column with binary data and I’m getting the following error:

java.lang.UnsupportedOperationException: Unsupported starrocks type, column name: test, data type: varbinary
    at com.starrocks.connector.spark.sql.schema.InferSchema.inferDataType(InferSchema.java:131)
    at com.starrocks.connector.spark.sql.schema.InferSchema.inferStructField(InferSchema.java:96)
    at com.starrocks.connector.spark.sql.schema.InferSchema.inferSchema(InferSchema.java:75)
    at com.starrocks.connector.spark.sql.StarRocksTableProvider.inferSchema(StarRocksTableProvider.java:65)
    at org.apache.spark.sql.execution.datasources.v2.DataSourceV2Utils$.getTableFromProvider(DataSourceV2Utils.scala:90)
    at org.apache.spark.sql.DataFrameWriter.getTable$1(DataFrameWriter.scala:280)
    at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:2xx)

Are `VARBINARY`/`BINARY` columns unsupported by design in the Spark connector today? Are there any plans to add support in the future?

Is there a specific reason you want binary data into the DWH? Some prefer to design with an ID or path to a Blob (S3/GCS) storage to keep the analytic layer more performant

I have a use case where I have pre-calculated HLLs in parquet and I want to upload them to the DWH