I have a real time table and sometimes because of ...
# troubleshooting
s
I have a real time table and sometimes because of upstream process going wrong, pinot ingests the same data again. So, basically some duplicate data. If we know when the upstream went bad, is there a way to identify segments created after a certain point of time and delete them through apis?
s
A very manual process (but can probably be scripted) is to download segment metadata for each segment and delete those in which the offset range contains the offending rows. Note that by removing a segment, you will also remove the other rows -- not just the offending rows. There is no way to remove single rows in Pinot.