Here’s a concise summary of what I did and what I found. What I set up • Verified both repos are present from the same org: ◦ OpenHands (V0): /workspace/project/OpenHands ◦ software-agent-sdk (V1): /workspace/project/software-agent-sdk • Built agent-sdk; confirmed LiteLLM is installed and importable in the SDK’s .venv. • Verified LiteLLM Gemini transform behavior locally using the SDK’s .venv. What happened with LLMNoResponseError in V0 (OpenHands) • Purpose: Handle the case where LiteLLM returns a normal ModelResponse with no choices (observed with Gemini). • Where we raise: ◦ On non-native function-calling reconstruction if resp.choices is empty: ▪︎ openhands/llm/llm.py#L349-L356 ▪︎ https://github.com/OpenHands/OpenHands/blob/main/openhands/llm/llm.py#L349-L356 ◦ General check: raise if choices missing/empty: ▪︎ openhands/llm/llm.py#L369-L374 ▪︎ https://github.com/OpenHands/OpenHands/blob/main/openhands/llm/llm.py#L369-L374 • Retry behavior: We bump temperature from 0.0 to 1.0 on retries only for LLMNoResponseError: ◦ openhands/llm/retry_mixin.py#L35-L55 ◦ https://github.com/OpenHands/OpenHands/blob/main/openhands/llm/retry_mixin.py#L35-L55 • Exception text references Gemini (documenting observed source): ◦ openhands/core/exceptions.py#L100-L110 ◦ https://github.com/OpenHands/OpenHands/blob/main/openhands/core/exceptions.py#L100-L110 • Introduction/related commit: ◦ https://github.com/OpenHands/OpenHands/commit/0ab9d97f2 • Direct answer to “did our code create it when LiteLLM returned a regular ModelResponse or when it raised a 500?”: ◦ V0 created LLMNoResponseError when LiteLLM returned a normal ModelResponse with empty/missing choices. It did not create this error on 500s; 500s were retried but without the temperature bump (bump is specific to LLMNoResponseError). What we did in V1 (agent-sdk) • We preserved the same behavior: ◦ Raise when choices missing/empty (inside retry boundary so bump applies): ▪︎ openhands-sdk/openhands/sdk/llm/llm.py#L534-L542 ▪︎ https://github.com/OpenHands/software-agent-sdk/blob/f3c0c19cd134fbda84e07f152897a6d61e1e46c5/openhands-sdk/openhands/sdk/llm/llm.py#L534-L542 ◦ Also raise in non-native FC post conversion: ▪︎ openhands-sdk/openhands/sdk/llm/mixins/non_native_fc.py#L60-L67 ▪︎ https://github.com/OpenHands/software-agent-sdk/blob/f3c0c19cd134fbda84e07f152897a6d61e1e46c5/openhands-sdk/openhands/sdk/llm/mixins/non_native_fc.py#L60-L67 ◦ Temperature bump on retry only for LLMNoResponseError: ▪︎ openhands-sdk/openhands/sdk/llm/utils/retry_mixin.py#L53-L66 ▪︎ https://github.com/OpenHands/software-agent-sdk/blob/f3c0c19cd134fbda84e07f152897a6d61e1e46c5/openhands-sdk/openhands/sdk/llm/utils/retry_mixin.py#L53-L66 ◦ Exception text references Gemini: ▪︎ openhands-sdk/openhands/sdk/llm/exceptions/types.py#L46-L54 ▪︎ https://github.com/OpenHands/software-agent-sdk/blob/f3c0c19cd134fbda84e07f152897a6d61e1e46c5/openhands-sdk/openhands/sdk/llm/exceptions/types.py#L46-L54 • Tests explicitly exercising this: ◦ Commit adding tests: https://github.com/OpenHands/software-agent-sdk/commit/53c0162e ◦ File: tests/sdk/llm/test_llm_no_response_retry.py ◦ Temperature bump assertion example: https://github.com/OpenHands/software-agent-sdk/blob/53c0162e/tests/sdk/llm/test_llm_no_response_retry.py#L99-L107 PR #1121 context (agent-sdk) • PR: Handle InternalServerError with choices=None to enable temperature bumping: ◦ https://github.com/OpenHands/software-agent-sdk/pull/1121 • Motivation: Some LiteLLM versions/providers raise InternalServerError when choices is None before returning a ModelResponse, so our current bump path (LLMNoResponseError) isn’t triggered. • As of current HEAD (f3c0c19c), mapping still treats InternalServerError as LLMServiceUnavailableError (no temp bump): ◦ openhands-sdk/…/exceptions/mapping.py#L43-L48 ◦ https://github.com/OpenHands/software-agent-sdk/blob/f3c0c19cd134fbda84e07f152897a6d61e1e46c5/openhands-sdk/openhands/sdk/llm/exceptions/mapping.py#L43-L48 Does this happen in reality? • Yes. We validated with the SDK’s .venv that LiteLLM’s Gemini transformer returns ModelResponse with choices=[] when Google returns empty candidates (no exception thrown by LiteLLM in that path). That triggers our LLMNoResponseError: ◦ Local reproduction (observed): ModelResponse choices len: 0 (transform_response call). • LiteLLM code shows the streaming path starts with choices=[] and only fills if candidates exist (so empty choices are normal to handle): ◦ Upstream file: https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py • PR #1121 addresses the complementary real-world path where newer LiteLLM validates and raises InternalServerError before we see a response, so the bump needs to be enabled by mapping that specific error to LLMNoResponseError. Conclusions • V0 created LLMNoResponseError for normal ModelResponse with empty choices (not for 500s). Retry bumped temperature from 0.0 to 1.0 only for this error. • V1 preserved this behavior and added tests. We did not lose behavior. • The code addresses real cases: Gemini via LiteLLM can return choices=[]; also, some LiteLLM versions may raise InternalServerError for choices None. PR #1121 is a sensible hardening to ensure the temperature bump applies in both paths. Task tracker status • t1: done; t2: done; t3: done; t4: done; t5: done. No code changes were made. If you want, I can: • Update the SDK to implement the #1121 mapping (if not already merged), add a focused test for that path, and open a PR; or • Keep this analysis local and proceed to your next request. _{View full conversation}

The commit that added InternalServerError to the retry exceptions in OpenHands is: • Commit: 0ab9d97f2 “fix(llm): retry on InternalServerError and Timeout; handle Gemini returns len(choices) < 1” ◦ https://github.com/OpenHands/OpenHands/commit/0ab9d97f2 • Associated PR: #7713 ◦ https://github.com/OpenHands/OpenHands/pull/7713 _{View full conversation}

Answer: No — in the agent-sdk’s current .venv, LiteLLM does not raise InternalServerError when Gemini returns empty or None choices. It either: • Returns a normal ModelResponse with choices=[], or • Raises VertexAIError for HTTP errors from the provider. What I checked (in your .venv) • File: /workspace/project/software-agent-sdk/.venv/lib/python3.12/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py ◦ transform_response → _transform_google_generate_content_to_openai_model_response: ▪︎ Always sets model_response.choices = [] and only fills choices when _candidates is present; otherwise returns with choices still empty. Observed in the printed sections around lines 1488–1725. ◦ Streaming: ModelResponseStream is created with choices=[] by default (around line 2209). ◦ Error handling: async_completion and completion catch httpx.HTTPStatusError and raise VertexAIError with the HTTP status code, not InternalServerError (around 1958–1971 and 2063–2072). • Searched the installed litellm package in the .venv: ◦ No occurrences of InternalServerError in the Gemini implementation or the broader litellm package in this environment. Implication • In this environment, LiteLLM returns ModelResponse(choices=[]) for Gemini when there are no candidates. Your SDK then raises LLMNoResponseError based on that empty choices response (not because LiteLLM raised a 500). HTTP errors surface as VertexAIError. _{View full conversation}

Summary of work • Verified the actual behavior in the agent-sdk .venv of LiteLLM for Gemini: ◦ Empty/missing candidates return a normal ModelResponse with choices=[] (no InternalServerError). ◦ Non-200 HTTP responses are raised as VertexAIError (not InternalServerError). ◦ Confirmed there are no InternalServerError references in the installed LiteLLM Gemini implementation. • Conclusion ◦ Given the current environment, PR #1121’s mapping of InternalServerError → LLMNoResponseError would not trigger for the real Gemini-empty-choices path we see today. Our existing behavior (raise LLMNoResponseError on choices=[] and bump temperature on retry) already covers the real-world case. • Posted a structured comment on PR #1121 with evidence and links: ◦ Comment link: https://github.com/OpenHands/software-agent-sdk/pull/1121#issuecomment-3513663016 • What’s in the PR comment ◦ Introduced myself as OpenHands-GPT-5. ◦ Clear markdown with headings and bold. ◦ Evidence links to LiteLLM Gemini code in BerriAI/litellm: ▪︎ transform_response and downstream conversion show choices is set to [] if no candidates. ▪︎ Streaming path initializes ModelResponseStream(choices=[]). ▪︎ HTTP errors are raised as VertexAIError, not InternalServerError. ◦ Context link to the V0 PR that introduced InternalServerError retry and LLMNoResponseError on empty choices: ▪︎ OpenHands PR #7713: https://github.com/OpenHands/OpenHands/pull/7713 ◦ Recommendation to focus on today’s observable behavior unless a reproducible InternalServerError-for-no-choices case is demonstrated. No code changes were made in repositories. _{View full conversation}