hey I have this query that i m issuing in pinot `select from Apache Pinot #troubleshooting

hey I have this query that i’m issuing in pinot `s...

Luis Fernandez

10/08/2021, 1:14 PM

hey I have this query that i’m issuing in pinot

select * from ads_metrics where user_id=x and serve_time >= 1633651200

when I use this query like this the

numEntriesScannedInFilter

shoots up quite considerably if I don’t use serve_time I get 0 anyone knows why that may be? I currenly have a rangeindex in the

serve_time

column and an invertedIndex + partitioning on the user_id

Luis Fernandez

10/08/2021, 1:16 PM

Copy code

You should also think about partitioning the incoming data based on the dimension most heavily used in your filter queries.

that’s what i see in the documentation about high

numEntriesScannedInFilter

but i’m already partitioning by the user_id

Kishore G

10/08/2021, 2:11 PM

We have a new version of range index that will reduce the latency @Richard Startin ^^

Richard Startin

10/08/2021, 2:38 PM

hi @Luis Fernandez you can try the new range index by setting

"rangeIndexVersion" : 2

"tableConfig"

you'd have to build off OSS master though

Richard Startin

10/08/2021, 2:39 PM

there's no automatic migration path yet because we want to get some feedback first before we start automating the upgrade

Mayank

10/08/2021, 7:20 PM

BTW, this is available in latest master and will be part of the next 0.9.0 release.

Luis Fernandez

10/11/2021, 8:34 PM

thank you very much for your insight

Luis Fernandez

10/11/2021, 8:34 PM

we are currently pulling from here

Luis Fernandez

10/11/2021, 8:34 PM

Copy code

"apachepinot/pinot:release-0.8.0-jdk11"

Luis Fernandez

10/11/2021, 8:34 PM

so we would basically have to change that?