Hello team, while ingesting pre-computed ThetaSket...
# general
s
Hello team, while ingesting pre-computed ThetaSketch column and storing them as
BYTES
in the realtime table, what does this error mean
Copy code
ERROR [LLRealtimeSegmentDataManager_aud_2__0__0__20240326T1804Z] [aud_2__0__0__20240326T1804Z] Caught exception while transforming the record: org.apache.pinot.spi.stream.StreamDataDecoderResult@326d20b3
java.lang.RuntimeException: Caught exception while transforming data type for column: maid_sketch
Caused by: java.lang.IllegalArgumentException: Value: AQMDAAA6zJOaaqWQa2cKTg== is not Hex encoded
This is generated through Scala (Spark) -
Copy code
{
  "timestamp": 1707121902000,
  "zid": 124421,
  "bd": "lorem",
  "maid_sketch": "AQMDAAA6zJMcsZ8XKakAZQ==",
  "segid": 9
}
The relevant Scala/Spark code-
Copy code
val sketch = UpdateSketch.builder().build()
sketch.update(maid)
sketch.compact().toByteArray()
Schema-
Copy code
{
  ...
  "name": "maid_sketch",
  "dataType": "BYTES"
  ...
}