Hello team 👋
We are facing an issue related to data ingestion and segment unavailability, would appreciate your thoughts on how is this approached for backfill / regular ingestion flows.
We have a Hadoop job that does a segment creation and metadata push for ingestion. Currently, the metdata push done brings the segment ingested ONLINE immediately in ideal state but not yet in external state since the servers take a few seconds to few minutes to download all the segments and there is a segment unavailability during that time.