Hi folks, I see that segments for a particular table are not being relocated to offline servers from realtime servers. For the particular table, I can see that

Relocation failed for table

as segments are in error state. We have used the Reset API for all the segments which were in BAD state. But trying to understand why this error occurs? (No table config change was made from our end and segments were being relocated successfully earlier).

2022/08/08 16:22:18.328 INFO [PinotLLCRealtimeSegmentManager] [grizzly-http-server-1] Committing segment metadata for segment: span_event_view_1__91__4093__20220808T1558Z
--
java.lang.IllegalStateException: Found segments in ERROR state
	at org.apache.pinot.controller.helix.core.rebalance.TableRebalancer.isExternalViewConverged(TableRebalancer.java:556) ~[pinot-all-0.9.1-jar-with-dependencies.jar:0.9.1-f8ec6f6f8eead03488d3f4d0b9501fc3c4232961]
	at org.apache.pinot.controller.helix.core.rebalance.TableRebalancer.waitForExternalViewToConverge(TableRebalancer.java:498) ~[pinot-all-0.9.1-jar-with-dependencies.jar:0.9.1-f8ec6f6f8eead03488d3f4d0b9501fc3c4232961]
	at org.apache.pinot.controller.helix.core.rebalance.TableRebalancer.rebalance(TableRebalancer.java:361) ~[pinot-all-0.9.1-jar-with-dependencies.jar:0.9.1-f8ec6f6f8eead03488d3f4d0b9501fc3c4232961]
	at org.apache.pinot.controller.helix.core.relocation.SegmentRelocator.lambda$processTable$0(SegmentRelocator.java:96) ~[pinot-all-0.9.1-jar-with-dependencies.jar:0.9.1-f8ec6f6f8eead03488d3f4d0b9501fc3c4232961]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
2022/08/08 17:10:25.590 ERROR [SegmentRelocator] [restapi-multiget-thread-2037] Relocation failed for table: span_event_view_1_REALTIME

Can you check the table debug endpoint in Swagger to see if it shows any exceptions on why the segment went into ERROR state. You can also grep the segment name in the server log.