Apache Pinot

but my suggestion is to not depend on that but use it as an optimization

yes i can add ordered ...so basically if the documents are for example stock_id, value and i have to group by "stock_id" the order of that group should be correct

hmm ok, so if i should not depend on that is better to not optimize it that way..

<@UDRJ7G85T> for each documents i need to loop on my ranges array, and more sparse are the input data and more ranges i will need to add inside that list...so then for each document i must loop on big range list..

or maybe create something that prevent me looping on ranges, maybe a direct access to the range i need based on key..but i do not know i have to investigate more

start with a map&lt;key, List&lt;Point&gt;&gt;

when the size of List get more than T (threshold), optimize it

or optimize it when the total number of points become more than T

<@UDRJ7G85T> yes i could do that but i do not think there is a way to filter by key no? i mean if with "key" you mean the timestamp how can i then undersatnd the timestamp for the max value and the timestamp for the min value

i must find a way to avoid looping the List or Map

that logic wont work with multiple maxima and minima

see this: <https://paste.ofcode.org/6NbViasxWR8Ljj5Qu6AVhu>

it works. It generate the result i expected and it works with completely random points

before adjust that code for Pinot i would like to optimize it, and as i told you more sparse the points are and more Ranges in the list i must to add

this is a good start but can be simplified

i have seend that it creates less or more 28k ranges

with huge i mean calling it many times takes time

in PInot, you will be given an array of 5k or 10k

ar eyou talking about segments? segments of 5-10 ?

lets try it out, we can optimize it later when we find perf issues