Hi, realtimeToOffline task tried to process some segments that were deleted and moved to Deleted Segments folder, in this case, the following error occurred. What should I do if I want Pinot to resume processing other segments and ignore those deleted segments? org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 404 (Not Found) with reason: “Segment telemetrics_data__0__0__20220408T0702Z or table telemetrics_data not found in /var/pinot/controller/data/telemetrics_data/telemetrics_data__0__0__20220408T0702Z”

the realtimeToOffline will not pick segments to move, unless they were in the ideal state. So it looks like the retention is set too aggressive? and the reatimeToOffline is not able to catch up?

@User, I think my test here was a little complex. The realtimeToOffline task failed due to large bucket window and limited resource. Then I modified retention from 10 days to 2 days, so segments older than 2 days were deleted. Then the task failed with the data not found error and it repeated returning the same error.

hi Alice, we could check the logs for some clues. In Pinot controller logs, you should be able to find some logs about the task scheduling (like by this line). There should be some task configs in those logs, and those task configs are used to kick off the processing on minion worker. I’d wonder if the list of input segments as in the task config are still in the table? We can also check the task states recorded in ZK to dig further as next.