Hi team. I have ran a cluster in pinot with deep_store. Segments were going in the S3 smoothly, the retention provided at that time was 7 days. the server crashed due to heavy load on server during festival. We decided to reduce the retention from 7 to 4 days. After doing that I reloaded all the segments. How long will it take the pinot to resume ingestion? Basically how long will it take to delete older segments?
@Xiang Fu@Mayank
m
Mayank
08/18/2025, 1:13 PM
You didn’t need to reload segments. You can call retention explicitly via api
r
Rajat
08/18/2025, 1:13 PM
which API?
m
Mayank
08/18/2025, 1:18 PM
Check swagger for periodic task api
x
Xiang Fu
08/18/2025, 2:05 PM
Usually controller retention manager should kick in within 5mins after you restart controller
r
Rajat
08/18/2025, 2:41 PM
It didn't start. So I had to re-ingest the data again
Rajat
08/18/2025, 2:41 PM
the server was crashed maybe thats why
Rajat
08/18/2025, 2:41 PM
server disk was 100% full
x
Xiang Fu
08/18/2025, 2:44 PM
controller should remove data from zk and deep store
Xiang Fu
08/18/2025, 2:44 PM
but if your server disk is full, you need to clean it up
r
Rajat
08/18/2025, 3:07 PM
yeah thats why I started the ingestion again
a
Aditya Verma
08/18/2025, 7:30 PM
hi @Mayank@Xiang Fu
Just few doubts on this thread.
I see but what if I need the data from inception for my use case ?
in that case I wont be able to setup a Data retention policy .
Also deleting data from server ?
Doesn’t server in Pinot queries data from deep store where the controller pushes it after realtime Kafka ingestion and then the server loads the data to its volume for querying from broker ?
m
Mayank
08/18/2025, 7:54 PM
Answered in dm. But answering here as well. Not specifying retention means infinite retention. Servers store local copy of data for fast serving, deepstore is for DR