Saif Ali Khan
03/26/2024, 6:40 PMBYTES in the realtime table, what does this error mean
ERROR [LLRealtimeSegmentDataManager_aud_2__0__0__20240326T1804Z] [aud_2__0__0__20240326T1804Z] Caught exception while transforming the record: org.apache.pinot.spi.stream.StreamDataDecoderResult@326d20b3
java.lang.RuntimeException: Caught exception while transforming data type for column: maid_sketch
Caused by: java.lang.IllegalArgumentException: Value: AQMDAAA6zJOaaqWQa2cKTg== is not Hex encoded
This is generated through Scala (Spark) -
{
"timestamp": 1707121902000,
"zid": 124421,
"bd": "lorem",
"maid_sketch": "AQMDAAA6zJMcsZ8XKakAZQ==",
"segid": 9
}
The relevant Scala/Spark code-
val sketch = UpdateSketch.builder().build()
sketch.update(maid)
sketch.compact().toByteArray()
Schema-
{
...
"name": "maid_sketch",
"dataType": "BYTES"
...
}