This message was deleted.

Segments

table is the one which gets used for the available segments (with USED=1) USED=0 are not used and can be deleted). The

pendingSegments

table consists of temporary entries that are used to coordinate segment allocation for streaming ingestion so that multiple tasks do not try to create the same segment. If there is no ingestion running we can delete the entries from the pending segments table, Large number of records here could cause long metadata queries as well.

id recommend also enabling druid.coordinator.kill.pendingSegments.on=true

As per document default is true, but we have seen some occurrences, where it is not done successfully, not sure on the reason :(

The coordinator runs a duty to clean up pending segments that are older than a day. Do you see any errors related to the cleanup from

KillStalePendingSegments

? If you see a legitimate issue, please file a GitHub issue or a PR to fix it 🙂 But as Bibek notes, pending segments only matter in the context of a running task, so it should be safe to delete stale entries.