Hi 👋 I’m wondering if retentionTimeValue is inclusive or not? (If I was to write a query with an equivalent filter, would I use

<

or

<=

?

I’d recommend using explicit filter for the time period that you are interested in the query. Retention is a periodic job, and it may be a few hours before the data actually is deleted. For instance, if you are interested in 28 days data, you may chose to have retention as 30 days, and have an explicit time filter for 28 days in your query.

The query I’m writing is to create the holding table to compare counts to what pinot has (determined by the retentionTimeValue) so it needs to match exactly

To answer your original question, records will be deleted only after they are older than

retentionTime

. But as I mentioned, records won’t be deleted exactly when retentionTime is hit, but in the next periodic job. As long as you are aware of that, we are good.

Ok yes 👍 thanks!

Also, if there are segments in which there is at least ONE record that is new, the segment won't be deleted if there are records in the segment that are several years old. So, it is always good to use a time filter as mentioned by mayank

Ok thanks! One more question: Is retentionTime date or timestamp based? pure date logic of 2022-06-14 - 2020-06-06 = 738 days but if it’s really now (which includes timestamp) - 2020-06-06 might make these records too old

It goes by retention time unit

Ok, so ours is set to DAYS so its just the date logic