Soon
08/28/2025, 2:25 PMAs explained in the forward index section, a column that is both sorted and equipped with a dictionary is encoded in a specialized manner that serves the purpose of implementing both forward and inverted indexes. Consequently, when these conditions are met, an inverted index is effectively created without additional configuration, even if the configuration suggests otherwise.
We have a column that is dictionary enabled and configured as sorted in realtime table. We have also set autoGeneratedInvertedIndex and createInvertedIndexDuringSegmentGeneration as true in the table config. However we are not seeing inverted index being used in the explain plan query. Would inverted index also be configured in the table config to see it in effect?Soon
08/28/2025, 2:26 PMsorted and autoGeneratedInvertedIndex and createInvertedIndexDuringSegmentGeneration is trueMayank
Soon
08/29/2025, 6:09 PMExecution Plan
PinotLogicalAggregate(group=[{}], agg#0=[COUNT($0)])
PinotLogicalExchange(distribution=[hash])
PinotLogicalAggregate(group=[{}], agg#0=[COUNT()])
LogicalFilter(condition=[AND(=($9, 4), =($10, 1523255), =($24, 53), >=($8, 20250827), <=($8, 20250828))])
LogicalTableScan(table=[[default, table_REALTIME]])Soon
08/29/2025, 6:10 PMMayank
select count(*) from <table> where <sortedColumn> = <xxx>Soon
08/29/2025, 6:27 PMExecution Plan
PinotLogicalAggregate(group=[{}], agg#0=[COUNT($0)])
PinotLogicalExchange(distribution=[hash])
PinotLogicalAggregate(group=[{}], agg#0=[COUNT()])
LogicalFilter(condition=[=($24, 53)])
LogicalTableScan(table=[[default, unified_events_v2_REALTIME]])Mayank
Soon
08/29/2025, 6:27 PMSoon
08/29/2025, 6:28 PM{
"requestId": "604985615000334402",
"brokerId": "x",
"offlineSystemActivitiesCpuTimeNs": 0,
"realtimeSystemActivitiesCpuTimeNs": 0,
"offlineThreadCpuTimeNs": 0,
"realtimeThreadCpuTimeNs": 0,
"offlineResponseSerializationCpuTimeNs": 0,
"realtimeResponseSerializationCpuTimeNs": 0,
"offlineTotalCpuTimeNs": 0,
"realtimeTotalCpuTimeNs": 0,
"numSegmentsPrunedByBroker": 0,
"numSegmentsPrunedByServer": 48,
"numSegmentsPrunedInvalid": 0,
"numSegmentsPrunedByLimit": 0,
"numSegmentsPrunedByValue": 0,
"explainPlanNumEmptyFilterSegments": 0,
"explainPlanNumMatchAllFilterSegments": 0,
"numServersQueried": 4,
"numServersResponded": 4,
"numDocsScanned": 17416655,
"numEntriesScannedInFilter": 1610336252,
"numEntriesScannedPostFilter": 0,
"numSegmentsQueried": 9958,
"numSegmentsProcessed": 9910,
"numSegmentsMatched": 9910,
"numConsumingSegmentsQueried": 48,
"numConsumingSegmentsProcessed": 0,
"numConsumingSegmentsMatched": 0,
"minConsumingFreshnessTimeMs": 1756455179799,
"totalDocs": 1654179622,
"numGroupsLimitReached": false,
"timeUsedMs": 1908,
"numRowsResultSet": 1,
"segmentStatistics": [],
"traceInfo": {},
"brokerReduceTimeMs": 0,
"exceptions": [],
"resultTable": {
"rows": [
[
x
]
],
"dataSchema": {
"columnNames": [
"count(*)"
],
"columnDataTypes": [
"LONG"
]
}
},
"maxRowsInJoinReached": null,
"maxRowsInOperator": null,
"maxRowsInWindowReached": null,
"stageStats": null,
"partialResult": false,
"resultObjects": [
{
"count(*)": x
}
]
}Mayank
"numEntriesScannedInFilter": 1610336252,Mayank
Soon
08/29/2025, 6:32 PMMayank
Soon
08/29/2025, 6:33 PMMayank
Soon
08/29/2025, 6:34 PMSoon
08/29/2025, 6:34 PMSoon
08/29/2025, 6:34 PMMayank