Is there anyway to clean up old segments files from the server file system. Do all servers need all segments locally or should we look to enable tiered storage.

Yes, to completely remove old segments from the server set the retentionTimeUnit and retentionTimeValue settings. No, not all servers need all of the segments locally. Use Tiered Storage only if you need to occasionally query older segments.