Hello - we are exploring storing metrics at higher granularities by rolling up the data at lower granularities. Ex: 1s metrics rolled up and stored at 1 min granularity. Does pinot support percentile aggregations ?
m
Mayank
12/08/2021, 9:33 PM
Yes, Pinot supports percentile, as well as approximate percentile (using TDigest). For your case, since you are rolling up you might explore percentileTDigest as it is an additive function.
s
Suraj
12/09/2021, 12:00 AM
thank you will explorer that !
Suraj
12/13/2021, 7:35 AM
@User is it possible to support percentiles as part of pre-aggregation i.e. during ingestion ? The docs say only SUM is supported
m
Mayank
12/13/2021, 2:38 PM
Currently only sum is supported. Do you have a lot or rows that have some dimension values that will get aggregated?
s
Suraj
12/13/2021, 6:59 PM
yes we basically want to store latencies over a longer term. Ex: hourly, daily, weekly percentiles (95, 99, 99.99 etc)