How can the test tell it's a comment update or a config 1 liner change that can take down the entire facebook? If the test infrastructure isn't smart enough to detect dependencies, always rerunning the test will be the safest bet. That being said, I think a good balance would be run the test when we open a PR and before we want to merge a PR. I don't know if there is an event for the latter.