Hi team I have a few fields that are arrays of byte arrays B Apache Pinot #troubleshooting

Hi team, I have a few fields that are arrays of by...

Gerrit van Doorn

07/21/2022, 11:30 PM

Hi team, I have a few fields that are arrays of byte arrays (BYTES), I made them multi value by setting

"singleValueField": false

in the schema but I’m confronted by the following error in the server:

Copy code

java.lang.UnsupportedOperationException: null
        at org.apache.pinot.segment.local.realtime.impl.dictionary.BytesOnHeapMutableDictionary.index(BytesOnHeapMutableDictionary.java:45) ~[pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
        at org.apache.pinot.segment.local.indexsegment.mutable.MutableSegmentImpl.updateDictionary(MutableSegmentImpl.java:538) ~[pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
        at org.apache.pinot.segment.local.indexsegment.mutable.MutableSegmentImpl.index(MutableSegmentImpl.java:486) ~[pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
        at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.processStreamEvents(LLRealtimeSegmentDataManager.java:550) [pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
        at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.consumeLoop(LLRealtimeSegmentDataManager.java:420) [pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
        at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:598) [pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
        at java.lang.Thread.run(Thread.java:829) [?:?]

Is this related to https://github.com/apache/pinot/issues/8635 ? NOTE: I’m not ingesting json

Gerrit van Doorn

07/21/2022, 11:37 PM

and on query time I see:

Copy code

[
  {
    "message": "QueryExecutionError:\njava.lang.IllegalStateException: Unsupported value type: BYTES for multi-value column\n\tat org.apache.pinot.core.common.RowBasedBlockValueFetcher.createFetcher(RowBasedBlockValueFetcher.java:83)\n\tat org.apache.pinot.core.common.RowBasedBlockValueFetcher.<init>(RowBasedBlockValueFetcher.java:32)\n\tat org.apache.pinot.core.operator.query.SelectionOnlyOperator.getNextBlock(SelectionOnlyOperator.java:97)\n\tat org.apache.pinot.core.operator.query.SelectionOnlyOperator.getNextBlock(SelectionOnlyOperator.java:40)",
    "errorCode": 200
  }
]

Gerrit van Doorn

07/21/2022, 11:44 PM

I see it’s not supported, I could probably use https://github.com/apache/pinot/blob/4548783928c92340849de908262ebb2a50f1e286/pino[…]cal/realtime/impl/dictionary/StringOnHeapMutableDictionary.java instead

Kishore G

07/22/2022, 12:52 AM

is it really byte array or string array

Kishore G

07/22/2022, 12:52 AM

what operations are you planning on this column

Open in Slack

Previous Next