https://pinot.apache.org/ logo
p

Pedro Silva

05/21/2021, 10:48 AM
Hello, How is someone meant to configure a json index by a string field that may sometimes be missing? I'm getting the following exception:
Copy code
java.lang.IllegalStateException: Cannot flatten value node: null
        at shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:518) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
        at org.apache.pinot.spi.utils.JsonUtils.flatten(JsonUtils.java:246) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
        at org.apache.pinot.core.realtime.impl.json.MutableJsonIndex.add(MutableJsonIndex.java:71) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
        at org.apache.pinot.core.indexsegment.mutable.MutableSegmentImpl.addNewRow(MutableSegmentImpl.java:631) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
        at org.apache.pinot.core.indexsegment.mutable.MutableSegmentImpl.index(MutableSegmentImpl.java:475) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
        at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.processStreamEvents(LLRealtimeSegmentDataManager.java:497) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
        at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.consumeLoop(LLRealtimeSegmentDataManager.java:402) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
Because a given field comes with null default value:
Copy code
"inputForUiControls" : "null",
It's schema definition is:
Copy code
dimensionFieldSpecs: [,...., {
      "name": "inputForUiControls",
      "dataType": "STRING",
      "maxLength": 2147483647
}]
Is the intended method to fix this situation to specify the default value as:
"defaultNullValue": "{}"
?
j

Jackie

05/21/2021, 6:19 PM
Hi Pedro, this issue has been fixed in the current master branch
@Xiang Fu Do we have a recent release that Pedro can try with?
x

Xiang Fu

05/21/2021, 6:29 PM
latest should have it
👍 1
p

Pedro Silva

05/21/2021, 6:31 PM
Thank you both, I will take a look next week