https://pinot.apache.org/ logo
e

Elon

08/05/2020, 11:01 PM
Also, noticed that we can't do text_match until the text index is created, looks like the latency could be up to 30 mins
s

Sidd

08/05/2020, 11:17 PM
Are the text_match queries returning empty results or taking long time to return response?
e

Elon

08/05/2020, 11:17 PM
Empty results, let me retry now, just to make sure
s

Sidd

08/05/2020, 11:20 PM
empty results could be due to the refresh lag.. realtime index reads the index snapshot and a background thread refreshes the snapshot periodically
how many realtime segments are there that are having text index and how many columns with text index per segment
the refresh threshold is configurable but currently it is hardcoded to 10ms
so that shouldn't lead to empty results
with refresh happening every 10ms, ideally you should see increasing number of hits/matches
also, what is the 30min latency?
e

Elon

08/05/2020, 11:23 PM
So 2 columns for text index
for the latency, do you mean the jmx counter?
s

Sidd

08/05/2020, 11:25 PM
No, I wanted to understand if you meant that queries are taking 30mins to return response
e

Elon

08/05/2020, 11:25 PM
Oh, no:) They return super fast
s

Sidd

08/05/2020, 11:25 PM
you said "looks like the latency could be up to 30 mins"
ok got it
we need to improve the lag I guess