Ravikumar Maddi
03/19/2021, 11:26 AM{
"schemaName": "eventflowstats",
"dimensionFieldSpecs": [
{
"name": "_index",
"dataType": "STRING"
},
{
"name": "_type",
"dataType": "STRING",
"maxLength": 5
},
{
"name": "_id",
"dataType": "STRING"
},
{
"name": "_source.aExpIds",
"dataType": "INT",
"singleValueField": false
}
]
"dateTimeFieldSpecs": [
{
"name": "_source.sDate",
"dataType": "LONG",
"format": "1:SECONDS:SIMPLE_DATE_FORMAT:SECONDS:SIMPLE_DATE_FORMAT",
"granularity": "1:DAYS"
}
]
}
My Table Config like this:
{
"tableName": "mytable",
"tableType": "REALTIME",
"tenants": {},
"segmentsConfig": {
"timeColumnName": "_source.sDate",
"timeType": "MILLISECONDS",
"segmentPushType": "APPEND",
"replicasPerPartition": "1",
"retentionTimeUnit": "DAYS",
"retentionTimeValue": "1"
},
"tableIndexConfig": {
"loadMode": "MMAP",
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.consumer.type": "lowLevel",
"stream.kafka.topic.name": "mytopic",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"stream.kafka.hlc.zk.connect.string": "localhost:2191/kafka",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.zk.broker.url": "localhost:2191/kafka",
"stream.kafka.broker.list": "localhost:19092"
}
},
"metadata": {
"customConfigs": {}
}
}
And Data like this:
{
"_index": "dhfkdfkdsjfk",
"_type": "_doc",
"_id": "68767677989hjhjkhkjh",
"_source.aExpIds": [
815850,
815857,
821331
],
"_source.sDate": "2021-01-04 00:00:00"
}
I check all logs, I did not find any exceptions. But data is not appearing in Pinot controller portal.Xiang Fu
event-count-stats-topic
you are producing to and the topic in your table configs are not same "stream.kafka.topic.name": "mytopic",
Ravikumar Maddi
03/19/2021, 3:09 PMXiang Fu
dateTimeFieldSpecs": [
{
"name": "_source.sDate",
"dataType": "LONG",
"format": "1:SECONDS:SIMPLE_DATE_FORMAT:SECONDS:SIMPLE_DATE_FORMAT",
"granularity": "1:DAYS"
}
Ravikumar Maddi
03/19/2021, 3:11 PMXiang Fu
"format": "1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd HH:mm:ss",
Ravikumar Maddi
03/19/2021, 3:21 PMXiang Fu