Does Pinot automatically delete the generated indices from the servers after deleting a segment? I'm running into an issue where I would delete the segment through the REST API, but it leaves the index files under PinotServer/index behind. The indices built up over time from my test tables and now the servers are out of disk space

The indices should be in the same folder as the segment, and removed along with the segment. Can you please check what files are left over on the servers?

It seems that all the segments are still remaining in PinotServer/index. I think my cluster somehow ended up in a weird state where it shows something like 1000/0 segments in the controller UI for that table.

Can you please check the idealstate and external view of the table?

The ideal state is empty. The external view actually shows all of the segments as OFFLINE

The external view still exist?

Yeah it looks like the external view still exists. I don't see any errors in the server logs. I think this issue happened when I tried to reset all of the segments but it timed out. After the failed reset, it left all of the segments OFFLINE. I then tried to delete them and now it is in the current state.

Can you search if there are state transition from OFFLINE to DROPPED logged on the server side? If not, then it might be controller/zookeeper issue

Ah I do see a bunch of transitions from OFFLINE->DROPPED. What does this mean in terms of the current state of the cluster?

Which

current state

were you referring to?

I just meant the state of that table in general. It's been almost a week and it looks like the segment files are still there. I think the process of deleting the segment files got stuck? Is there a recommended way of cleaning up the segments/table?