https://pinot.apache.org/ logo
Channels
Powered by
m

Mayank

03/31/2021, 12:30 AM
What's the data type for 
webConferenceId
?
c

Charles

03/31/2021, 12:31 AM
LONG
m

Mayank

03/31/2021, 12:31 AM
Is this a hybrid table?
c

Charles

03/31/2021, 12:32 AM
message has been deleted
It should be a realtime table
m

Mayank

03/31/2021, 12:33 AM
You have 90 days retention, so my guess is there's an offline component. But that is ok
At the face of it, this seems like a bug
Trying to understand what might be causing it
c

Charles

03/31/2021, 12:33 AM
So it should be a bug?
m

Mayank

03/31/2021, 12:33 AM
Can you try with 
webSiteId
?
c

Charles

03/31/2021, 12:34 AM
String Type is ok, let me try again
j

Jackie

03/31/2021, 12:34 AM
Can you please paste the entire table config?
c

Charles

03/31/2021, 12:35 AM
schema< 
{
“schemaName”:“realtime_sjc_wmequality_report”, “dimensionFieldSpecs”:[ { “name”:“webexSiteName”, “dataType”:“STRING” }, { “name”:“webexConferenceId”, “dataType”:“LONG” }, { “name”:“webexSiteId”, “dataType”:“LONG” }, { “name”:“correlationId”, “dataType”:“STRING” }, { “name”:“metadataOsType”, “dataType”:“STRING” }, { “name”:“metadataOsVersion”, “dataType”:“STRING” }, { “name”:“metadataBrowserType”, “dataType”:“STRING” }, { “name”:“metadataClientType”, “dataType”:“STRING” }, { “name”:“metadataClientVersion”, “dataType”:“STRING” }, { “name”:“metadataHardwareType”, “dataType”:“STRING” }, { “name”:“metadataNetworkType”, “dataType”:“STRING” }, { “name”:“audioMainReportTransportType”, “dataType”:“STRING” }, { “name”:“videoMainReportTransportType”, “dataType”:“STRING” }, { “name”:“day”, “dataType”:“STRING” } ], “metricFieldSpecs”:[ { “name”:“systemAverageCPU”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“processAverageCPU”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“osBitWidth”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“cpuBitWidth”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“audioMainReportRxE2eLostPercent”, “dataType”:“FLOAT”, “defaultNullValue”:0 }, { “name”:“audioMainReportRxE2eJitter”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“audioMainReportTxHbhLostPercent”, “dataType”:“FLOAT”, “defaultNullValue”:0 }, { “name”:“audioMainReportTxHbhJitter”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“audioMainReportRxHbhLostPercent”, “dataType”:“FLOAT”, “defaultNullValue”:0 }, { “name”:“audioMainReportRoundTripTime”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“videoMainReportRxE2eLostPercent”, “dataType”:“FLOAT”, “defaultNullValue”:0 }, { “name”:“videoMainReportRxE2eJitter”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“videoMainReportTxHbhLostPercent”, “dataType”:“FLOAT”, “defaultNullValue”:0 }, { “name”:“videoMainReportTxHbhJitter”, “dataType”:“LONG”, “defaultNullValue”:0 }, { “name”:“videoMainReportRxHbhLostPercent”, “dataType”:“FLOAT”, “defaultNullValue”:0 }, { “name”:“videoMainReportRoundTripTime”, “dataType”:“LONG”, “defaultNullValue”:0 } ], “dateTimeFieldSpecs”:[ { “name”:“timestamp”, “dataType”:“STRING”, “format”“1MILLISECONDSSIMPLE DATE FORMATyyyy-MM-dd’T’HHmmss.SSS’Z’“, “granularity”“1MILLISECONDS” } ] }
table: 
{
“tableName”:“realtime_sjc_wmequality_report”, “tableType”:“REALTIME”, “segmentsConfig”:{ “timeColumnName”:“timestamp”, “timeType”:“DAYS”, “retentionTimeUnit”:“DAYS”, “retentionTimeValue”:“90”, “segmentPushType”:“APPEND”, “segmentAssignmentStrategy”:“BalanceNumSegmentAssignmentStrategy”, “schemaName”:“realtime_sjc_wmequality_report”, “replication”:“2”, “replicasPerPartition”:“2” }, “tenants”:{}, “tableIndexConfig”:{ “loadMode”:“MMAP”, “streamConfigs”:{ “streamType”:“kafka”, “stream.kafka.consumer.type”:“LowLevel”, “stream.kafka.topic.name”:“sj1_mqa_telemetry_wmequality_report”, “stream.kafka.decoder.class.name”:“org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder”, “stream.kafka.consumer.factory.class.name”:“org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory”, “stream.kafka.broker.list”“10.241.89.1309092", “realtime.segment.flush.threshold.time”: “24h”, “realtime.segment.flush.threshold.size”: “300M”, “stream.kafka.consumer.prop.auto.offset.reset”: “largest” }, “invertedIndexColumns”:[ “webexSiteName”, “webexConferenceId”, “webexSiteId”, “correlationId”, “metadataOsType”, “metadataBrowserType”, “metadataClientType”, “metadataHardwareType”, “metadataNetworkType”,“audioMainReportTransportType”,“videoMainReportTransportType”,“day” ], “sortedColumn”:[“audioMainReportRxE2eLostPercent”,“audioMainReportRxE2eJitter”] }, “metadata”{“customConfigs”{}} }
j

Jackie

03/31/2021, 12:38 AM
Do you have time for a quick zoom? Need to try more queries to identify the issue
c

Charles

03/31/2021, 12:39 AM
webSiteId is ok, also LONG type
j

Jackie

03/31/2021, 12:45 AM
How about 
select * from table where websiteId = 8049967 limit 1000
? Want to see if the missing conferenceId is returned here
c

Charles

03/31/2021, 12:45 AM
ok
m

Mayank

03/31/2021, 12:46 AM
Yeah, that is what I also meant earlier
c

Charles

03/31/2021, 12:46 AM
message has been deleted
No response?
message has been deleted
The status of segments, seems normal
j

Jackie

03/31/2021, 12:52 AM
Sorry, should be 
webexSiteId = 8049967
c

Charles

03/31/2021, 12:53 AM
ok
message has been deleted
has response
j

Jackie

03/31/2021, 12:55 AM
Let's try 
select * from realtime_sjc_wmequality_report where webexConferenceId = '189852985506937900' limit 1000
first to rule out the possibility of compilation problem
c

Charles

03/31/2021, 12:56 AM
message has been deleted
No response
j

Jackie

03/31/2021, 1:08 AM
@Charles Can you join this zoom? We can try some queries together to track down the problem https://us04web.zoom.us/j/77675017805?pwd=N2FwM3lPSlcxUTVjMU4zZ0FxTGhNQT09
Sure
c

Charles

03/31/2021, 1:11 AM
ok
Joined
Powered by