Hey @quick-window-91705, I noticed that the review automation was failing silently, probably due to the 10 minute limit being hit. I opened this PR to make better cleanup when we hit the termination limit so we could ideally post a message that review failed in that case: https://github.com/OpenHands/automation/pull/204 I’m not an expert on this part of the codebase at all though. I asked OpenHands to do a live test and see what it comes back with, but would you mind if I turned this over to you? If the PR is totally off base then you could also just discard it.

Did a bit of digging, and it seems as though the proposed solution is making the status update of the run immediately consistent as opposed to eventually consistent (and only for our presets), which I think is not as necessary We do, for a good UX, need a way to invoke a custom callback for automations that fail or finish successfully (e.g post review failed comment), and a way to configure the timeout limits for automations LMK if that seems reasonable and I can close the PR mentioned above accordingly

Yeah, makes sense. I think the custom callback is what I wanted to achieve, but obviously did not 😅