My concern is helix managing millions of segments, 99% of which are never touched.
k
Kishore G
11/02/2020, 4:53 PM
Note that we can also enhance Helix
n
Noah Prince
11/02/2020, 4:54 PM
And really, which segments are lazy loaded is an important part of the cluster state. It’s useful from a monitoring perspective.
n
Neha Pawar
11/02/2020, 4:58 PM
the unloading could also be done by the same periodic task which is responsible for moving from tier to tier (assuming you are planning to use the tiered storage design). Based on some flag in the tier config, for storageType=deepStore
s
Slackbot
11/02/2020, 5:31 PM
This message was deleted.
x
Xiang Fu
11/07/2020, 1:04 AM
this tutorial uses segment uri push, since you already on newest version, segment metadata push will be much better