https://pinot.apache.org/ logo
Join Slack
Powered by
I’m getting `Caught exception while transforming t...
# troubleshooting
t
I’m getting 
Caught exception while transforming the record:
when ingesting realtime stream, it’s a 
java.lang.ClassCastException: null
exception, looks like it’s casting a null value to some type, is this a bug?
m
Without asking more details could be a bug. Are you using your own custom transform? If so, then the bug may be in your custom code. Else, if you can file an issue with more details that would be really helpful.
t
We are ingesting json message from kafka, there some field is missing in the json payload, but present in the table schema, I can’t see many details from the log.
m
Can you paste the full stack trace from the exception?
t
There is no stacktrace printed in the log.
m
Just this: 
java.lang.ClassCastException: null
?
t
Copy code
2021/04/27 02:58:04.025 ERROR [LLRealtimeSegmentDataManager_pad_interaction__0__154__20210426T2358Z] [pad_interaction__0__154__20210426T2358Z] Caught exception while transforming the record: {
  "fieldToValueMap" : {
    "server_time" : 0,
    "app_version" : "",
    "action_message" : "",
    "page_type" : "",
    "is_whale_test" : 0,
    "download_startTime" : 0,
    "sensor_sn" : "WIA201923230F435CDA",
    "app_version_code" : 0,
    "download_url" : "",
    "download_status" : "",
    "download_duration" : 0,
    "action" : "iot_device_data",
    "action_value" : [ "MQTT", "edge_server", 3413 ],
    "model" : "",
    "auto_run_app" : "",
    "showcase_rule_id" : "",
    "download_endTime" : 0,
    "device_sn" : "WRK2019242EE9408E28",
    "company_id" : "3640348656460353536",
    "device_id" : "",
    "group_name" : "家乐福-相宜本草-上海龙之梦购物中心-中岛-1",
    "download_retryCount" : 0,
    "os_version" : "",
    "count" : 1,
    "action_count" : 1,
    "shop_name" : "家乐福-相宜本草-上海龙之梦购物中心",
    "actor_sex" : "",
    "shop_id" : "4324843702787114240",
    "app_name" : "",
    "component" : "",
    "download_appName" : "",
    "event_id" : "",
    "group_id" : "6841bafd-076a-4e16-90a0-650b7ee95aea",
    "template_duration_time" : 0,
    "company_name" : "家乐福",
    "template_id" : "",
    "time" : 1619492283856,
    "template_skuid" : "",
    "actor_faceid" : ""
  },
  "nullValueFields" : [ "server_time", "app_version", "action_message", "page_type", "download_startTime", "app_version_code", "download_url", "download_status", "download_duration", "model", "auto_run_app", "showcase_rule_id", "download_endTime", "device_id", "download_retryCount", "os_version", "count", "actor_sex", "app_name", "component", "download_appName", "event_id", "template_duration_time", "template_id", "template_skuid", "actor_faceid" ]
}
java.lang.ClassCastException: null
that’s the full log
m
In the record printed, are the fields from schema just missing, or are they empty/null?
t
They are missing, I checked message from kafka, so I think missing value will be parsed as null.
m
Missing in every record, or just some?
If just some, perhaps a better behavior would be to treat them as null and substitute by default null value. In that case, can you file an issue?
t
I think there are some column missing in every message, is this the expected behavior if I have missing columns?
m
If in every record then perhaps it is a misconfiguration to begin with? And if so, may be better to fail and force correcting. What do you think?
t
I think in this situation missing column should be filled with default value, the missing columns are not static, some record missing column a, another column might missing column b, maybe this should be handled by the json parser?
m
Ok, if they are not static, then may be fill with default null value. Can you file an issue?
t
Sure, I will file an issue for this.
Open in Slack
PreviousNext
Powered by