Hi Team, I was looking at ID_SET and IN_SUB_QUERY ...
# general
s
Hi Team, I was looking at ID_SET and IN_SUB_QUERY provisions in pinot for handling subqueries referring the below video:

https://www.youtube.com/watch?v=HryANqHnTQk&t=686s

Here I have few questions: 1. Is the ID_SET only supported for integer values? 2. Is there support for alphanumeric strings? Any pointers would be helpful
k
@User ^^
j
@User ID_SET supports all data types. For non-integer types (types other than INT and LONG), it stores the values in a bloom filter
The
expectedInsertions
and
fpp
is configurable for the bloom filter to tune the accuracy. You may read more here: https://docs.google.com/document/d/1s6DZ9eTPqH7vaKQlPjKiWb_OBC3hkkEGICIzcd5gozc/edit?usp=sharing
j
Not trying to criticize harshly, but from my experiments (single use case), ID_SET was much (much) slower on non-integer types Maybe it's still worth giving it a shot for your use case, but think about the parameters @User, they make a big difference