anmol
05/25/2025, 5:20 AMSTRING[]), but I'm facing segment consumption errors.
Here’s what I’ve done so far:
• Pinot version: 1.0+
• In Kafka, the message contains a field like:
"txn_notification_event_list": "[\"ZORBLAT\",\"QUIXANO\",\"FLUMPTION\",\"WIBBLEX\",\"SNOOFLE-VORTEX\",\"PLINKO-FUND-SWAP\",\"ZOOGLE\",\"FX-TRANSMOG\"]"
I tried the following transformConfig in the table config:
Config 1 :
{
"columnName": "txn_notification_event_list_array",
"transformFunction": "jsonExtractArray(txn_notification_event_list, '$', 'STRING')"
}
Config 2 :
"transformConfigs": [
{
"columnName": "txn_notification_event_list_array",
"transformFunction": "jsonFormatArray(JSONPARSE(txn_notification_event_list))"
}
]
And defined this in the schema:
{
"name": "txn_notification_event_list_array",
"dataType": "STRING",
"singleValueField": false
}
• The segment ends up in ERROR state on both servers. No segmentSize, no consumerInfo, and no errorInfo shown in the Pinot UI.
I've verified that the Kafka messages are correctly formatted as stringified arrays, but I’m not sure if Pinot is parsing this properly or if something’s misconfigured in my schema/table setup.
Would appreciate any help or pointers!
Thanks!Mayank
Mayank
stringToArray