This message was deleted.
# generals
Slackbot
02/08/2024, 4:33 PMThis message was deleted.
k
Kyle Hoondert
02/08/2024, 4:53 PMThe whitelist is set using the Coordinator Dynamic properties https://druid.apache.org/docs/latest/configuration/#dynamic-configuration
j
JRob
02/08/2024, 4:54 PMHow do I specify "all" datasources?
k
Kyle Hoondert
02/08/2024, 4:58 PMIf
killDataSourceWhitelistKyle Hoondert
02/08/2024, 4:58 PMj
JRob
02/08/2024, 4:59 PMAh, okay, that's not documented. Hmm..
Our coordinator is not issuing kill tasks and I assumed it was because we had not set the whitelist.
k
Kyle Hoondert
02/08/2024, 4:59 PMActually it is
j
JRob
02/08/2024, 5:04 PMHmm. I see:
JRob
02/08/2024, 5:04 PMk
Kyle Hoondert
02/08/2024, 5:06 PMoh - like you didn’t have kill tasks turned on? Right, they are not on by default
j
JRob
02/08/2024, 5:23 PMIt is turned on but still no kill tasks 😢
k
Kyle Hoondert
02/08/2024, 5:25 PMYou have something like this set?
Copy code
druid.coordinator.kill.pendingSegments.on=true
druid.coordinator.kill.on=true
druid.coordinator.kill.period=PT4H
druid.coordinator.kill.durationToRetain=PT24H
druid.coordinator.kill.maxSegments=200000j
JRob
02/08/2024, 5:36 PMI don't have
Copy code
druid.coordinator.kill.pendingSegments.on=truedruid.coordinator.kill.bufferPeriodPT6HJRob
02/08/2024, 6:00 PMConfig that wasn't working (kill tasks not running):
Copy code
$ cat runtime.properties |grep druid.coordinator.kill
druid.coordinator.kill.on=true
druid.coordinator.kill.period=PT1H
druid.coordinator.kill.durationToRetain=P1D
druid.coordinator.kill.maxSegments=10000 Copy code
$ cat runtime.properties |grep druid.coordinator.kill
druid.coordinator.kill.on=true
druid.coordinator.kill.period=PT1H
druid.coordinator.kill.durationToRetain=P1D
druid.coordinator.kill.bufferPeriod=PT6H
druid.coordinator.kill.maxSegments=10000JRob
02/08/2024, 6:03 PMVery first kill interval:
Copy code
"interval": "2023-12-30T22:00:00.000Z/2024-02-07T15:00:00.000Z",JRob
02/14/2024, 5:06 PM@Kyle Ahn I have come back to check after applying these settings and I can see that Deep Storage is still not being cleaned up. I believe we have an issue with version 28.0.1 not cleaning Deep Storage. I did not have an issue before upgrading.
JRob
02/14/2024, 5:09 PMThe introduction of
druid.coordinator.kill.bufferPeriodJRob
02/14/2024, 5:33 PMMoved to #C0309C9L90D
https://apachedruidworkspace.slack.com/archives/C0309C9L90D/p1707931446776669