another question: if I import data into pinot thru the standalone job, and say I have a retention of 2 years, will it remove data as it ages from the data I imported? or how does that work with the retention manager.

Retention is based on the age of data as defined by time column, and not the time from when it was pushed. This works as background task.

so if time column time > retention time, retention manager will remove yes?

It is a periodic task (a few hours, don’t recall if 1 or 6hrs)

default 6h, config: https://docs.pinot.apache.org/basics/components/controller#retentionmanager

thank you both 🙏