https://pinot.apache.org/ logo
Join Slack
Powered by
Pinot logging has a bug <here> (mismatching number...
# troubleshooting
s
Pinot logging has a bug here (mismatching number of arguments) that truncates the error details.
This is preventing me from finding out why Pinot does not like the JSON record I sent to it through Kafka. Can someone fix this and suggest an alternative debugging solution for me?
n
e
doesn’t need 
{}
in error()
💀 1
what’s the log you’re seeing?
s
Untitled
Untitled
The JSON I sent is valid and does not contain new lines.
n
the exception should be logged after the record. Can you share a few more lines of this log
s
So this does look like a truncated log to me, no?
no more lines
catch Exception is an anti-pattern, right?
would be helpful at least knowing what type of Exception happened.
more helpful if Exception message is output first.
n
i reproduced this scenario by emitting an event that will fail the transform phase
Copy code
2020/09/15 14:25:44.927 INFO [CallbackHandler] [ZkClient-EventThread-29-localhost:2181] 29 END:INVOKE /PinotCluster/INSTANCES/Server_10.1.10.51_8098/MESSAGES listener:org.apache.helix.messaging.handling.HelixTaskExecutor@27381587 type: CALLBACK Took: 0ms
2020/09/15 14:25:44.927 INFO [CallbackHandler] [CallbackHandler-AsycSubscribe-Singleton] Subscribing to path:/PinotCluster/INSTANCES/Server_10.1.10.51_8098/MESSAGES took:0
2020/09/15 14:25:50.024 INFO [AbstractCoordinator] [transcript__0__0__20200915T2125Z] [Consumer clientId=consumer-4, groupId=] Discovered group coordinator localhost:9876 (id: 2147483647 rack: null)
2020/09/15 14:27:58.781 ERROR [LLRealtimeSegmentDataManager_transcript__0__0__20200915T2125Z] [transcript__0__0__20200915T2125Z] Caught exception while transforming the record: {
  "fieldToValueMap" : {
    "studentID" : 205,
    "firstName" : "Natalie",
    "lastName" : "Jones",
    "score" : 3.8,
    "timestampInEpoch" : 1571900400000,
    "gender" : "Female",
    "subject" : "Maths"
  },
  "nullValueFields" : [ ]
}
java.lang.NumberFormatException: For input string: "Maths"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_242]
	at java.lang.Integer.parseInt(Integer.java:580) ~[?:1.8.0_242]
	at java.lang.Integer.parseInt(Integer.java:615) ~[?:1.8.0_242]
	at org.apache.pinot.common.utils.PinotDataType$9.toInteger(PinotDataType.java:319) ~[classes/:?]
	at org.apache.pinot.common.utils.PinotDataType$5.convert(PinotDataType.java:201) ~[classes/:?]
	at org.apache.pinot.common.utils.PinotDataType$5.convert(PinotDataType.java:168) ~[classes/:?]
	at org.apache.pinot.core.data.recordtransformer.DataTypeTransformer.transform(DataTypeTransformer.java:112) ~[classes/:?]
	at org.apache.pinot.core.data.recordtransformer.CompositeTransformer.transform(CompositeTransformer.java:82) ~[classes/:?]
	at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.processStreamEvents(LLRealtimeSegmentDataManager.java:486) [classes/:?]
	at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.consumeLoop(LLRealtimeSegmentDataManager.java:398) [classes/:?]
	at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:532) [classes/:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
works as expected for me
it seems like you’re missing some log lines
s
thx for looking into this. I’ll see if there are other things that could truncate the log lines. Can we still show exception message first as that’s the most important and concise info?
Open in Slack
PreviousNext
Powered by