We hit a failure mode with OpenHands Automations where a preset/plugin automation triggered correctly, created a cloud conversation row, then failed before any conversation events were persisted. Observed behavior: - GitHub issue event matched the automation filter correctly. - Automation run created a sandbox and conversation ID. - Run failed with: Server error '500 Internal Server Error' for url 'http://localhost:60000/api/conversations/<id>/events' - Conversation export contained only meta.json, no events. - Conversation metadata had zero token usage/cost and no event history. - Manual dispatch of the same plugin-preset automation failed the same way, so it was not the GitHub webhook/event payload. - A prompt-preset automation using the same repo clone path completed successfully. Likely product improvements: 1. Surface the real plugin/setup exception. The user-facing run error should not be just

/events

500; it should include the failing setup/plugin-loading step and sanitized stdout/stderr. 2. Preserve setup logs in failed conversation exports. Exporting only

meta.json

makes this almost impossible to debug. 3. Add an integration test for

preset/plugin

automations with

repos

, GitHub plugin source,

repo_path

, and event trigger/manual dispatch. 4. If plugin loading fails, write an explicit error event into the conversation before failing the run. 5. Add a “validate automation” or “dry-run setup” path that checks repo clone, plugin load, MCP config, and agent startup before users wait for a webhook. 6. Docs should clarify when users should use

preset/prompt

+ repo

.agents/skills

versus

preset/plugin

. For this demo, prompt-preset with repo skills was more reliable and still supported event automations. 7. The run/conversation UI should show the generated automation type, entrypoint, setup status, sandbox status, and a direct link to sanitized run logs. Workaround that succeeded: - Switched from

preset/plugin

to

preset/prompt

. - Kept the repo in

repos

. - Put skills under

.agents/skills

. - Added a repo skill helper that calls the public MCP endpoint directly. - New event automation completed and opened a PR successfully. The main message: plugin-preset failure currently looks like a user auth/repo/event issue, but the actual problem is hidden inside automation runtime/plugin setup.

(I am building a demo where I want to go from local skill to event based automation in the cloud)