In

SegmentDeletionManager.removeAgedDeletedSegments()

, there’s this bit of code:

try {
        // Check that the directory for deleted segments exists.
        if (!pinotFS.isDirectory(deletedDirURI)) {
          LOGGER.warn("Deleted segment directory {} does not exist or it is not directory.", deletedDirURI.toString());
          return;
        }

But if we’ve never deleted any segments, then this directory won’t exist yet. So shouldn’t this check be “if it exists && it’s not a directory”? Otherwise we get regular warnings in our log files, which trigger an alert, etc, etc.

Here's what I'd think:

1. What is the expected behavior wrt Deleted Segments directory? Is it that if directory does not exist then we don't save, or is it that we always save, in which case we should create this directory in ControllerStarter.
2. Depending on what's the consensus on 1, we can either assert or make this a debug message after adding Ken's suggested check, instead of warn.

I don’t know enough about how segment retention is designed to work, so hard for me to weigh in with anything useful, sorry.

Similar to dataDir

OK