https://pinot.apache.org/ logo
Join Slack
Powered by
# inconsistent-perf
  • k
    its a simple fix
  • k
    we will submit the fix and will be great if you can test it
  • s
    Sure we will test it out.
  • s
    Can the fix be submitted today?
  • k
    @User can you take it up?
  • j
    @User Can you try with the branch in this pr: https://github.com/apache/incubator-pinot/pull/5331
  • j
    @User I added a threshold to switch between dictId-based and value-based as this could have big performance impact for low cardinality dictionary
  • j
    Let's evaluate it before merging
  • k
    sweet. yes, thats a great idea
  • s
    Sure will get back
  • s
    hey @User and @User tried the scenario with mentioned branch. Now its performing equally great with and without time range predicate!🎉
    🎉 1
  • k
    Thanks @User
  • s
    In case of realtime segments using offheap dictionaries, it should not take time to do a hash lookup. I suppose we were not using offheap here?
  • k
    We were using dictionary and that was the problem
  • k
    Since dictionary is not sorted in real-time, we had to scan the entire dictionary to find matching dictionary id’s
  • s
    Ah, ok. Each value in the dictionary is looked up.... got it
  • s
    One ques why would a query that is to be served by a completed segment also be affected by something in consuming seg?
  • k
    It does not
  • s
    @User ompleted segments have sorted dictionaries
  • s
    Do yuo have a sorted column setting also?
  • k
    Yes
  • s
    Yes got that but in our case v noticed that if there is any consuming seg then any further queries with range time stamp is also impacted
  • s
    Those further queries could be served from completed segment
  • k
    @User I am not following, are you referring to a test after the latest fix by Jackie or earlier tests?
  • s
    Earlier tests .. Before fix..
  • j
    @User Are you saying with a time filter time < T when consuming segment should not be included, you still see query getting impacted?
  • k
    the query was >T
  • s
    @User thanks for the fix. Ignore my question for now.. We are performing few tests and we will get back to you in a while..
  • s
    Sorry could not get back earlier
  • k
    @User has left the channel