https://docs.all-hands.dev/ logo
Join Slack
Powered by
<@U0BME18CF7S> there was a request for pr review 9...
# automations-debug
c
@fresh-machine-37269 there was a request for pr review 9 hours ago here: github.com/OpenHands/software-agent-sdk/pull/4918#… Identify it and tell me, why did it stall? just answer, don't do anything
👀 1
f
🤖 On it! View progress here: oss-agent-canvas.ngrok.dev/conversations/…
✅ Done! It was the OpenHands bot's own code-review run on PR #4918 (docs: remove stale 
run-eval
AGENTS reference). The comment you linked (
7:41:55Z
— "OpenHands is reviewing this PR") was just the start notification, not the review itself. Why it stalled: the review agent actually finished its analysis fine (concluded "no material findings, worth merging"), but it never got the review posted to GitHub. Plowing through its event log, it kept failing at the final "submit review" step for two reasons: 1. GitHub API rate limit — the bot's token (user ID 
182006032
) returned 
403 API rate limit exceeded
, so the POST of the review comment kept getting rejected. 2. Shell wrapper rejections — its multi-line heredoc submission commands were refused with "Cannot execute multiple commands at once", forcing repeated retries, which then collided with the rate-limit error. Net effect: it finished with the verdict in its own log but never successfully delivered the review, so the PR shows no bot review and appears stalled.
c
@fresh-machine-37269 ok I think we should approve it then
👀 1
f
✅ Done! Done — I approved PR #4918. ✅ The approval review (
state: APPROVED
, ID 
5157559900
) is now posted on the PR: github.com/OpenHands/software-agent-sdk/pull/4918#… I used the bot's own verdict from its stalled run — 🟢 LOW risk, documentation-only, "worth merging" — so the review matches what the earlier review agent had concluded but couldn't deliver.
c
@fresh-machine-37269 thanks! how about github.com/OpenHands/software-agent-sdk/pull/4204#… what happened there?
👀 1
f
✅ Done! Same story as #4918, essentially. Here's what happened on PR #4204 ("ci: enable API compliance and condenser test labels"): • The comment you linked (
07:43:04Z
— "OpenHands is reviewing this PR") was just the start notification for the latest review attempt on head commit 
5f4c9601
. • The review agent actually finished its analysis and generated a review: 🟢 no material code findings (CI-only change, condenser label routed through the existing matrix path), but it flagged one failing check — 
Validate PR description
 — because the PR's 
## Issue Number
says 
N/A
even though it references follow-up issue #4203 elsewhere. • But it stalled again at the submit step due to GitHub API rate limiting (
403 rate limit exceeded for user ID 182006032
) while trying to post the review / check run statuses. So the review never landed — the newest review on the PR is still from Aug 5, nothing from today's run. Net: the bot produced its review but never delivered it, so the PR sits open and blocked with 
Validate PR description
still failing. The actual blocker is a human-owned field — someone needs to fill in the 
## Issue Number
section (point it at #4203) and re-run the check.
c
please fix the pr description and I guess we can merge it if it doesn't affect anything but tests @fresh-machine-37269
👀 1
f
⚠️ The agent encountered a problem (status: stuck). Same story as #4918, essentially. Here's what happened on PR #4204 ("ci: enable API compliance and condenser test labels"): • The comment you linked (
07:43:04Z
— "OpenHands is reviewing this PR") was just the start notification for the latest review attempt on head commit 
5f4c9601
. • The review agent actually finished its analysis and generated a review: 🟢 no material code findings (CI-only change, condenser label routed through the existing matrix path), but it flagged one failing check — 
Validate PR description
 — because the PR's 
## Issue Number
says 
N/A
even though it references follow-up issue #4203 elsewhere. • But it stalled again at the submit step due to GitHub API rate limiting (
403 rate limit exceeded for user ID 182006032
) while trying to post the review / check run statuses. So the review never landed — the newest review on the PR is still from Aug 5, nothing from today's run. Net: the bot produced its review but never delivered it, so the PR sits open and blocked with 
Validate PR description
still failing. The actual blocker is a human-owned field — someone needs to fill in the 
## Issue Number
section (point it at #4203) and re-run the check.
c
@fresh-machine-37269 please fix the pr description
👀 1
f
⚠️ The agent encountered a problem (status: stuck). Same story as #4918, essentially. Here's what happened on PR #4204 ("ci: enable API compliance and condenser test labels"): • The comment you linked (
07:43:04Z
— "OpenHands is reviewing this PR") was just the start notification for the latest review attempt on head commit 
5f4c9601
. • The review agent actually finished its analysis and generated a review: 🟢 no material code findings (CI-only change, condenser label routed through the existing matrix path), but it flagged one failing check — 
Validate PR description
 — because the PR's 
## Issue Number
says 
N/A
even though it references follow-up issue #4203 elsewhere. • But it stalled again at the submit step due to GitHub API rate limiting (
403 rate limit exceeded for user ID 182006032
) while trying to post the review / check run statuses. So the review never landed — the newest review on the PR is still from Aug 5, nothing from today's run. Net: the bot produced its review but never delivered it, so the PR sits open and blocked with 
Validate PR description
still failing. The actual blocker is a human-owned field — someone needs to fill in the 
## Issue Number
section (point it at #4203) and re-run the check.
c
@fresh-machine-37269 why did this one fail? https://github.com/OpenHands/extensions/pull/547#issuecomment-5610412330 it happened in your environment. just answer me, don’t do anything yet.
👀 1
f
🤖 On it! View progress here: oss-agent-canvas.ngrok.dev/conversations/…
✅ Done! It failed in my environment because the PR-reviewer run's LLM call errored out — not because of anything in the PR itself. Specifically, the review agent was configured with model 
openai/gpt-5.6-sol
, and two things happened back-to-back: 1. The API call returned a 404 Not Found (
litellm.NotFoundError
), a non-retryable config error. 2. Shortly after, it hit a rate limit — 
usage_limit_reached
("The usage limit has been reached", team plan). Both of the duplicate review runs (two review conversations were spawned for this PR) died with the same two errors, which is why the bot posted the generic "⚠️ PR Reviewer encountered a problem … status: error" comment. The GitHub Actions checks on the commit were all green — the failure was purely the review agent's LLM/account, not the code under review.
c
@fresh-machine-37269 when does the limit reset? also, check the agent-sdk and automation repos, is there a PR open to add fallback LLM?
👀 1
f
✅ Done! When the limit resets: 
2026-09-15T01:33:22 UTC
— that's ~5 days from now (about 5 days, 1h 41m). Fallback LLM PRs: • software-agent-sdk: Yes — github.com/OpenHands/software-agent-sdk/pull/4917 "Fail over to fallback LLM immediately on hard quota exhaustion". Still open, not merged (not draft). • automation: No PR for fallback LLM specifically. The two closest matches are github.com/OpenHands/automation/pull/120 "Support named LLM profiles in automations" and github.com/OpenHands/automation/pull/223 "fix: capture active LLM profile from the correct users/me field" — both are about profile selection, not failover.
c
@fresh-machine-37269 /codereview 4917 please and post your feedback
👀 1
f
🤖 On it! View progress here: oss-agent-canvas.ngrok.dev/conversations/…
✅ Done! Code review posted to PR #4917. LGTM overall — the change is correct and well-scoped. I left a non-blocking comment covering: 1. A cosmetic type-annotation suggestion (
tenacity.retry_base
instead of 
Callable[..., Any]
on 
retry_exceptions
). 2. Note that the substring classifier is consistent with siblings and the patterns are sound. 3. Optional doc/test for the intentional exclusion of transient 
429 RESOURCE_EXHAUSTED
. Tests correctly cover the classifier (hard vs. transient 429), immediate failover (no retries), and the transient-still-retries regression guard. The checked-in 
glm.json
uses the same masked placeholder as 
fast.json
, so no secret concerns.
c
@fresh-machine-37269 fix the CI as best as you can please
👀 1
f
✅ Done! CI fixed and pushed to PR #4917. What was failing: the 
pre-commit
job's pyright step reported a type error in 
openhands-sdk/openhands/sdk/llm/llm.py:1055
— the 
lambda e: not is_quota_exhaustion_error(e)
predicate receives 
BaseException
but 
is_quota_exhaustion_error
was typed 
Exception
. There was also an import-ordering (ruff/isort) nit. Fixes (committed and pushed as 
60f23a7
): 1. Widened 
is_quota_exhaustion_error
signature to 
BaseException
(matches tenacity's predicate contract and the sibling classifiers' intent). 2. Reordered the 
tenacity
import below the 
litellm
block (auto-fixed by pre-commit). Verified locally: full 
pre-commit run --all-files
passes (pyright, ruff, pycodestyle, import rules), and the affected tests (
test_exception_classifier.py
, 
test_llm_fallback.py
) pass — 40/40. Note: I hit intermittent GitHub API rate-limit throttling while inspecting the two failing check runs, so I reproduced and fixed via the pre-commit output + local run rather than the exact failing-job logs. The changes are scoped to the two pre-commit findings; the SDK tests were already green locally.
c
@fresh-machine-37269 quota exceeded string is too wide, is it important for this particular error?
👀 1
f
🤖 On it! View progress here: oss-agent-canvas.ngrok.dev/conversations/…
⏳ Still working — this is taking longer than this run allows. Follow along at the link above; mention me again for an update.
2 Views
Open in Slack
PreviousNext
Powered by