Hi team, we have been facing an issues in writing ...
# general
s
Hi team, we have been facing an issues in writing the segments to miniò wrapped s3 and we have hard limits of 150TB and its reaching the hard limits - seems like data has’nt been deleted for so long Observed that issues started after I updated the retention. Does the attached retention deletes the data on s3 ? Or anything wrong ?
a
Hi Sai. Retention rules are meant to ask the historicals to drop segments. If I understand correctly your limits are being hit on deep storage due to segment data. Please try using kill tasks to delete unused segments (https://druid.apache.org/docs/latest/data-management/delete/#kill-task) Have you considered enabling segment auto kill?
s
Yes, I’ll have to initiate the kill tasks. But how can set the retention to initiate the kill tasks to delete the data on s3 ? Is above configuration do ?
Is above configuration do ?
No. Just to reiterate the configuration that you shared corresponds to load/drop of data on Druid historicals.
Kill task configuration (requires restart)
The kill configuration dictates how frequently to run kill tasks and how old the data must be before killing it
✅ 1
s
yes .it helps. Kill tasks were running fine until somepoint of time may be i observed a month ago. I still see kill tasks triggered by coordinator for other datasources and not for one major datasource that contributes higher data.
when you say restart the kill configuration. how can i acheive that ?
i already have kill related properties configured
a
If you already have them configured no restart is needed.
If you make the change to the coordinator runtime properties, the coordinators must be restarted, as you may already be aware