Hello, I’m having issue with

LIMIT

on a table with 4.5 millions of rows, When I’m doing this query:

SELECT * FROM datasource_609bc4f74e3c000300131110 ORDER BY "timestamp" ASC LIMIT 100000,10

I’m getting a result in ~2.5s, and I can see in the query response stats

totalDocs=4794306

which is fine But when I’m doing this one (offset 1 000 000 instead of 100 000):

SELECT * FROM datasource_609bc4f74e3c000300131110 ORDER BY "timestamp" ASC LIMIT 1000000,10

I’m getting no rows (which isn’t the expected behavior) in ~10s and the totalDocs is

569840

(which isn’t the expected behavior either?) I have an hybrid table with segmentPruning by time Do you have any idea why I’m having this kind of issue? Thank you

Hi, can you please check the other metadata within the query response? I suspect some servers timed out because of the high offset

Right, it seems servers timeouts, I haven’t thought of that, thank you

Unfortunately currently there is no index that can accelerate this query. What is the jvm setting for your servers? You might want to consider increasing the memory limit

We have 16G of heap and have around 32G available for nmaped file, do have any estimation of how much we should allocate ?

That should be enough

yep, right here https://github.com/apache/incubator-pinot/issues/6982