Hi team, I am running into an issue on Pinot 1.4.0 with a REALTIME upsert table when trying to use UpsertCompactMergeTask. Context I have a REALTIME upsert table with enableSnapshot=true, and I configured UpsertCompactMergeTask. However, the task does not generate any work for a subset of segments. These segments appear in the UI with: • segment.realtime.status = "IN_PROGRESS" • Helix/server state = OFFLINE • no segment.download.url • no segment.realtime.endOffset • no segment.total.docs Example segment metadata:

{
  "segment.realtime.numReplicas": "1",
  "segment.creation.time": "1717253407093",
  "segment.flush.threshold.size": "10000",
  "segment.realtime.startOffset": "4317",
  "segment.realtime.status": "IN_PROGRESS"
}

In the UI, the segment replica is shown as OFFLINE. Example segment name:

news__4__1__20240601T1450Z

These segments are also not present in deep store. Problem When I trigger UpsertCompactMergeTask, Pinot skips these segments because their download URL is empty. Controller logs:

WARN [UpsertCompactMergeTaskGenerator] Skipping segment news__4__1__20240601T1450Z for task as download url is empty
...
INFO [ServerSegmentMetadataReader] Retrieved validDocIds metadata for 0 segments from 1 server requests.
INFO [UpsertCompactMergeTaskGenerator] Finished generating 0 tasks configs for table: news_REALTIME

So the task cannot proceed because these segments are not considered mergeable. My understanding These look like old zombie consuming segments: • still marked as IN_PROGRESS • server state is OFFLINE • no deep store artifact • no download URL • apparently never committed as completed segments Because of that: • UpsertCompactMergeTask skips them • they cannot be reloaded from deep store • they look like stale metadata / orphaned realtime segments Questions 1. Is this a known issue in Pinot 1.4.0? 2. What is the recommended way to repair or clean up these old IN_PROGRESS + OFFLINE realtime segments when they no longer exist in deep store? 3. Should they be removed from metadata manually, or is there a supported repair workflow? 4. Is there any controller/server operation that should convert or clean up these stale segments automatically? Thanks !

seems to be a legacy realtime consumption issue. I would suggest to delete those kind of segments manually