Yupeng Fu
12/01/2020, 9:25 PMSELECT hour_start_timestamp_utc FROM downtime WHERE (secondsSinceEpoch > 1606247126) ORDER BY secondsSinceEpoch DESC, hour_start_timestamp_utc DESC LIMIT 1
it scans the past 1 week of data but return only 1 record.
since the table is large, it ends up scanning about 100 million records per query, and takes seconds
query output is like
{
"selectionResults": {
"columns": [
"hour_start_timestamp_utc"
],
"results": [
[
"2020-12-01 09:00:00"
]
]
},
"exceptions": [],
"numServersQueried": 9,
"numServersResponded": 9,
"numSegmentsQueried": 1059,
"numSegmentsProcessed": 1059,
"numSegmentsMatched": 18,
"numConsumingSegmentsQueried": 0,
"numDocsScanned": 142101504,
"numEntriesScannedInFilter": 0,
"numEntriesScannedPostFilter": 284203008,
"numGroupsLimitReached": false,
"totalDocs": 7374174837,
"timeUsedMs": 3522,
"segmentStatistics": [],
"traceInfo": {},
"minConsumingFreshnessTimeMs": 0
}
Mayank
Yupeng Fu
12/01/2020, 9:30 PMMayank
Yupeng Fu
12/01/2020, 9:31 PMXiang Fu
Ken Krugler
12/01/2020, 9:38 PMMayank
max
would still scan, unless we can somehow prune out segmentsYupeng Fu
12/01/2020, 9:43 PM"message": "QueryExecutionError:\njava.lang.NumberFormatException: For input string: \"2020-10-15 09:00:00\"\n\tat sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)\n\tat sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)\n\tat java.lang.Double.parseDouble(Double.java:538)\n\tat org.apache.pinot.core.segment.index.readers.StringDictionary.getDoubleValue(StringDictionary.java:58)\n\tat org.apache.pinot.core.operator.query.DictionaryBasedAggregationOperator.getNextBlock(DictionaryBasedAggregationOperator.java:66)\n\tat org.apache.pinot.core.operator.query.DictionaryBasedAggregationOperator.getNextBlock(DictionaryBasedAggregationOperator.java:43)\n\tat org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)\n\tat org.apache.pinot.core.operator.CombineOperator$1.runJob(CombineOperator.java:105)\n\tat org.apache.pinot.core.util.trace.TraceRunnable.run(TraceRunnable.java:40)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)\n\tat com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)"
Mayank
Xiang Fu
Yupeng Fu
12/01/2020, 9:45 PMXiang Fu
Yupeng Fu
12/01/2020, 9:46 PMMayank
Yupeng Fu
12/01/2020, 9:47 PMMayank
Yupeng Fu
12/01/2020, 9:48 PMMayank
Yupeng Fu
12/01/2020, 9:49 PMMayank
Yupeng Fu
12/01/2020, 9:49 PMMayank