My concern is helix managing millions of segments, 99% of which are never touched.

Note that we can also enhance Helix

And really, which segments are lazy loaded is an important part of the cluster state. It’s useful from a monitoring perspective.

the unloading could also be done by the same periodic task which is responsible for moving from tier to tier (assuming you are planning to use the tiered storage design). Based on some flag in the tier config, for storageType=deepStore

This message was deleted.

this tutorial uses segment uri push, since you already on newest version, segment metadata push will be much better