erik bergsten
04/22/2022, 2:28 PMRichard Startin
04/22/2022, 2:35 PMdatetimeconvert
there is no longer a dictionary to rely on, so one needs to be materialised on the fly, which slows the evaluation downdatetimeconvert
which pre-materialises the function at various granularites (hour, day, week etc.) precisely so group by queries can be acceleratederik bergsten
04/22/2022, 2:41 PMRichard Startin
04/22/2022, 2:48 PMerik bergsten
04/22/2022, 2:50 PMRichard Startin
04/22/2022, 2:51 PMerik bergsten
04/22/2022, 2:57 PMPinot will pre-generate one column per time granularity with forward index and range index. The naming convention is $${ts_column_name}$${ts_granularity}, e.g. Timestamp column ts with granularities DAY, MONTH will have two extra columns generated: $ts$DAY and $ts$MONTH.
I simply put a sorted-column: ["$ts$MONTH"] in my indexconfig ?Richard Startin
04/22/2022, 2:58 PMerik bergsten
04/22/2022, 3:12 PM"fieldConfigList": [
{
"name": "ts",
"encodingType": "DICTIONARY",
"indexType": "TIMESTAMP",
"indexTypes": [
"TIMESTAMP"
],
"timestampConfig": {
"granularities": [
"DAY",
"WEEK",
"MONTH",
"YEAR"
]
}
}
],
In the table config. Getting really good performance on time queries now!Richard Startin
04/22/2022, 3:21 PMJohan Adami
04/22/2022, 6:18 PMGetting really good performance on time queries now!back to the original 250ms or even better?
Xiang Fu
erik bergsten
04/23/2022, 7:06 AMXiang Fu
erik bergsten
05/10/2022, 6:15 PMXiang Fu
erik bergsten
05/13/2022, 2:11 PMXiang Fu
$ts$DAY
etcerik bergsten
05/23/2022, 2:54 PM"fieldConfigList": [
{
"name": "ts",
"encodingType": "DICTIONARY",
"indexTypes": [
"TIMESTAMP"
],
"timestampConfig": {
"granularities": [
"DAY",
"WEEK",
"MONTH",
"YEAR"
]
}
}
],
The result from /tables
"unrecognizedProperties": {
"/instanceAssignmentConfigMap": null,
"/tierConfigs": null,
"/tableIndexConfig/bloomFilterConfigs": null,
"/tableIndexConfig/starTreeIndexConfigs": null,
"/tableIndexConfig/segmentPartitionConfig": null,
"/tableIndexConfig/columnMinMaxValueGeneratorMode": null,
"/fieldConfigList": null,
"/ingestionConfig/filterConfig": null,
"/segmentsConfig/retentionTimeUnit": null,
"/segmentsConfig/retentionTimeValue": null,
"/segmentsConfig/crypterClassName": null,
"/segmentsConfig/peerSegmentDownloadScheme": null,
"/query/timeoutMs": null,
"/upsertConfig": null,
"/routing/segmentPrunerTypes": null,
"/routing/instanceSelectorType": null,
"/task": null,
"/quota/storage": null,
"/quota/maxQueriesPerSecond": null
},
Xiang Fu
erik bergsten
05/24/2022, 2:17 PMXiang Fu
Rong R
05/25/2022, 12:12 AM/
at the beginning. Did you escape the JSON payload correctly?