https://pinot.apache.org/ logo
#pinot-dev
Title
# pinot-dev
k

khush

03/22/2021, 5:23 AM
Hi, does pinot by default create sorted forward index on a column of type datetime?
m

Mayank

03/22/2021, 5:42 AM
Hi, no this is not the case.
Are you seeing any issues, or just curious? Trying to get the question behind the question.
k

khush

03/22/2021, 6:41 AM
Just curious to know
Also the intent is whether adding such a index will improve the query performance? Considering these values will be unique in all the rows. I was considering sorted forward index on raw values.
j

Jackie

03/22/2021, 5:47 PM
Currently we treat datetime column the same as regular columns except for retention management. Also, sorted index is always dictionary encoded, and cannot be applied directly to raw values
If the values are sorted within the segment, Pinot will detect that and generate sorted index automatically
k

khush

03/23/2021, 12:23 PM
My datetime field is sorted in a Kafka partition. So will pinot automatically create sorted index for it?
Also is there a way to verify if the index was created?
m

Mayank

03/23/2021, 2:13 PM
For real-time ingrstuons, sorted index generation happens within Pinot. You can look at the index metadata api via swagger to check if the segment is sorted
k

khush

03/23/2021, 2:21 PM
Okay. Thank you.
Tried using /segments/{tableName}/{segmentName}/metadata . Didn't get the index info. Is there any other api I should be using?
m

Mayank

03/23/2021, 6:01 PM
Oh my bad, that gives the ZK metadata probably. Do you have access to the segment directory?
If so, there's a metadata.properties file in there that should tell you the indexing info
k

khush

03/24/2021, 9:03 AM
Are we referring to this metadata?
m

Mayank

03/24/2021, 3:17 PM
Yes
k

khush

03/24/2021, 3:20 PM
Okay are there any other indexes created by default?
m

Mayank

11/09/2022, 6:46 AM
No indexes are not created by default.