https://pinot.apache.org/ logo
#general
Title
# general
r

Ronak

03/17/2021, 4:18 PM
I was exploring TEXT_MATCH functionality with pinot-0.7.0/0.6.0 and had configured one of the columns for it. Is there any configuration for the refresh time interval for the index - https://docs.pinot.apache.org/basics/indexing/text-search-support After enabling indexing (with
index type: Text
and
encoding type: RAW
) on the column and doing TEXT_MATCH, I was first getting an empty result, but after some time, I was getting the result. So, what is the initial delay for such a column to be searchable? Is any settings/configuration (e.g num of docs, indexed size, etc) for the same?
s

Sidd

03/17/2021, 4:26 PM
Hi Ronak, The refresh threshold isn't yet configurable. We can make it though. However, there was a bug I recently fixed. It was causing the lag. Just realized it's in my branch. Will create a PR
👍 3
r

Rishabh

03/17/2021, 5:18 PM
So by design/indexing there won’t be any significant lag?
r

Ronak

03/18/2021, 10:10 AM
@Sidd @Kishore G @Xiang Fu On similar lines, I was also exploring JSON index support with apache-pinot-0.7.0. In that context, I have few questions. • is JSON index supported on multi-value column? • Is there any plan in extending text_match kind of support on json value space? As shown in below example as Q1 doing term query on
message
field of json document.
Copy code
Table = json_doc_example
Example data with 3 rows:
id_col, json_doc_col
"1234", {"message": "too many retries"}
"1235", {"message": "error sql timeout"}
"1236", {"message": "too large filter"}

Q1: select * from json_doc_example JSON_MATCH(json_doc_col, 'message=''too''') : empty result -> can this return in future -> [1234, 1236] // or is it recommended to use both text index + json index for such use case on that column?
Q2 select * from json_doc_example JSON_MATCH(json_doc_col, 'message=''too lareg filter''') : returns -> [1236]
k

Kishore G

03/18/2021, 2:18 PM
Text is not supported for strings within json as of now. We can add that since we have the primitives.. please file an issue
r

Ronak

03/18/2021, 3:29 PM
👍 sounds good, I will file one.
@Kishore G @Xiang Fu I have filed this ticket - https://github.com/apache/incubator-pinot/issues/6752 @Sidd Can you point your PR related to
lag
related to
text_match
?