😂

Here’s my attempt at number 1, of making sure that issues are ready for development: github.com/OpenHands/OpenHands/pull/16449 It’d be great if I could get a review and get this merged so I can test if it’s working properly.

✅ Done! Done! Here's what I created: PR: github.com/OpenHands/extensions/pull/465 — adds the

qa-changes

automation catalog entry. How the automation works: • Trigger: GitHub

pull_request.opened

and

pull_request.ready_for_review

events • JMESPath filter (event-level):

!pull_request.draft && !icontains(pull_request.body, 'QA Agent') && glob(repository.full_name, '<repo>')

— only fires when the PR is not a draft, has no existing "QA Agent" section, and matches the configured repo • Prompt gate checks (agent-level): The prompt tells the agent to first verify (1) the PR is not a draft, (2) the PR's human author has more than 3 commits already in main (via GitHub API), and (3) no "QA Agent" section exists in the PR description. If any check fails, the agent stops without making changes — no conversation started. • Report delivery: Instead of posting a comment/review, the agent edits the PR description to append

---

followed by a

## QA Agent

section with the structured QA report. The prompt I used (adapted from the qa-changes skill's

prompt.py

, with the PR-description-editing tweak):

A pull request was opened or marked ready for review in {{form.repository}}. Before doing any QA work, run these gate checks and stop immediately (making no changes) if any fails:

1. The PR must not be a draft — it must be open for review. If the PR is a draft, stop.

2. The PR author (the human who opened it) must have more than 3 commits already merged into the {{form.ref}} branch. Use the GitHub API to count the author's commits in {{form.ref}}. If the count is 3 or fewer, stop without making any changes.

3. The PR description must not already contain a section named QA Agent (i.e., a header like ## QA Agent or ### QA Agent). If that section already exists, stop without making any changes or starting a conversation.

If all gate checks pass, proceed with the QA validation using the /qa-changes skill methodology:

• Phase 1: Understand the change. Read the PR diff, title, and description. Identify the goal of the PR. Classify every changed file. Form a hypothesis about what the PR should achieve.

• Phase 2: Set up the environment. Read the repo's bootstrap instructions, install dependencies, and build if needed. Note CI status but do not re-run tests.

• Phase 3: Exercise the changed behavior. Actually run the software the way a real user would — start servers, run CLI commands, make HTTP requests, open browsers. Do NOT run the test suite, linters, or code analysis. For bug fixes, reproduce the bug before and after the fix. Show before/after evidence.

• Phase 4: Report results.

IMPORTANT — how to post your report:

Do NOT post a new comment, review, or inline review comment. Instead, EDIT the PR description (the pull request body) to append your QA report at the very end, after a horizontal rule (---), under a new section titled QA Agent. Use the GitHub API (PATCH /repos/{owner}/{repo}/pulls/{pr_number} with a body field) to update the PR description. Append the following structure to the existing PR body:

———

QA Agent

{Your full QA report following the /qa-changes skill report format: verdict, summary, Does this PR achieve its stated goal? section, status table, collapsible evidence in details blocks, and issues found.}

Do not make any other changes to the PR description — only append the QA Agent section at the end. Do not post any comments or reviews.

About "disabled": I wasn't able to create the actual automation instance via the cloud API — the automation API key available to me is expired. Once PR #465 is merged, you can create the automation disabled by running:

# Create the automation
curl -X POST "<https://app.all-hands.dev/api/automation/v1/preset/prompt>" \
  -H "Authorization: Bearer $OPENHANDS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"QA Changes Agent - OpenHands/OpenHands","prompt":"...","repos":[{"url":"OpenHands/OpenHands","provider":"github","ref":"main"}],"trigger":{"type":"event","source":"github","on":["pull_request.opened","pull_request.ready_for_review"],"filter":"!pull_request.draft && !icontains(pull_request.body, '\''QA Agent'\'') && glob(repository.full_name, '\''OpenHands/OpenHands'\'')"}}'

# Then disable it (replace {id} with the returned ID)
curl -X PATCH "<https://app.all-hands.dev/api/automation/v1/{id}>" \
  -H "Authorization: Bearer $OPENHANDS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"enabled": false}'

The catalog entry in the manifest is ready to use from Agent Canvas once merged — it'll show up as "QA Changes Agent" in the Automations templates page.