This message was deleted.
# general
s
This message was deleted.
s
What are your retention rules? In general, the process goes something like this: • coordinator wakes up every minute (
druid.coordinator.period
) and checks for published/used segments and all the current retention rules • for each segment it evaluates which retention rule to apply • it prioritizes loading segments which have no presence within the target tier and then replicas • it generates load/drop commands for historicals The historicals read the load/drop commands from their load queue which has a max depth of maxSegmentsInNodeLoadingQueue. Each historical works independently to execute its load/drop commands using one or more threads based on each historical's config druid.segmentCache.numLoadingThreads To answer your question directly, AFAIK, the coordinator does not prioritize by tier but you can control the resources used by the historicals in each tier for segment loading.
j
Okay that answers my question, I was under the incorrect assumption that historicals coordinate their data loading to balance available segments. Since they all work independently I can just configure their settings individually by tier. My retention rules are setup in a way where there's no data overlap between both tiers as well as no replicas of any segment. Thanks for the help Sergio
👍 1
j
I use a loadByPeriod (P3M) for my hot tier followed by a loadForever to my cold tier.