Hello Is it possible to benefit from Pinot's Star-Tree index when performing aggregation queries on known time ranges ? For example, if I know in advance that I will get queries like this :

SELECT SUM(value) FROM values WHERE timeString BETWEEN '2021-01-01' AND '2021-01-08'

(ex: rolling week) Can some configuration of StarTree index precompute this sort of query ? (or even part of it) [I know this looks like a TSDB use case, but still, I'm hopeful 😄]

You could do that, I'd recommend first checking the perf of out-of-the-box Pinot.

Yep, actually I've tried it out of the box, and performance was already very good, but I was thinking ahead in case we need even more (or have greater volume of data) Good to know it's possible ! Thanks @User 🙂

Is this a realtime only use case?

Should be mostly batch in that case

I asked because you can also do some pre-aggregation in your batch job. In short, multiple options to explore

That would be outside Pinot, right ? Or is there some ingestion option for this ?

Good question - There's a pre-aggregate metrics option for RT tables to do it within Pinot. For batch, you could do it more efficiently outside.

You can simply add

timeString

into the star-tree split order and star-tree is able to handle the query

@User you're right, daily data

Then including

timeString

in the star-tree dimension split order should work well

I'll keep that in mind, thanks !

Yeah, the out-of-box should already give you a lot of mileage. But as Jackie said, if you need more you have this option.