Apache Pinot

from <https://docs.pinot.apache.org/operators/operating-pinot/tuning/routing#querying-all-segments>
```Currently, we have two different mechanisms to prune segments on the broker side to minimize the number of segment for processing before scatter-and-gather.```
But I only see `partitioning` . What is the second mechanism?

I see `Bloom Filter for Dictionary`  mechanism in release 0.4.0. <https://docs.pinot.apache.org/v/release-0.4.0/operators/operating-pinot/tuning/routing#bloom-filter-for-dictionary>. Is this mechanism deprecated?

Anything else I can do for an offline refresh table?

Those techniques mostly come into play for really high throughput at low latency requirements. What’s your throughput/latency requirement, I want to ensure if you need it.

1k+ qps with ~100 ms. <@U03QVGLAGJV> can add more details.

we did see performance benefits even at much lower qps with broker pruning

```"routing": {
    "segmentPrunerTypes": [
      "time","partition"
    ]
  },```

But then you also have to partition the data right? Or was this despite this?

In the current setup we have data partitioned but also have time pruning

Right. In our case the user builds the query and that's why there isn't an easy way to partition the data for us. <@U03QVGLAGJV> is still looking into it. In the meantime I was hoping to explore other mechanisms without partitioning. Thanks Luis for sharing your experience.

ohh I think u can still try to add time _i think_ like do routing with time only

please correct me if i am wrong <@U03QVGLAGJV>. we don't have time as one of the field. i'm guessing that you have time similar to realtime table to use as a segment boundary.

There is a eventtimestamp column. Though , I am not sure how that will help in our queries for the offline table.

It doesn't help if we are not using it in the query

Right it only helps to know the segment boundary