(even on this thread)

@fresh-machine-37269 two things please: • first, was PR 5098 linked above created by an automation that is defined in your VM? look at their names if any does a periodic sweep of models • second, i left a few comments on pr, please fix accordingly

✅ Done! Both parts are done. Here's the summary. 1. Yes — PR #5098 was created by an automation running in this VM. It's the "LLM Model List Update" automation (

LLM Model List Update

, ID

80cee119…

), which runs on a cron schedule (

0 9 * * 3,6

UTC = Wednesdays & Saturdays at 09:00). Its job is exactly a _periodic sweep of models_: it upgrades LiteLLM, diffs

model_cost

+ OpenRouter against

verified_models.py

in

software-agent-sdk

, and opens a (draft) PR for any newly released models. The 2026-09-16 run is what produced PR #5098. 2. Fixed your three review comments on the PR and pushed to the branch (

add-new-llm-models-20260916

): •

qwen3-max-thinking

→ removed ("too old") •

qwen3.6-max-preview

→ removed ("superseded") •

gpt-5-pro

/

gpt-5.2-pro

→ removed (kept

-pro

from 5.4 up:

gpt-5.4-pro

,

gpt-5.5-pro

remain) The removals were applied consistently in both `VERIFIED_OPENAI_MODELS`/`VERIFIED_QWEN_MODELS` and the curated

VERIFIED_OPENHANDS_MODELS

. Net result is now 9 additions (was 17), tests pass (

test_model_features

,

test_model_list

,

test_llm_router

all green), and

ruff

is clean. The PR remains a draft.

I mean, it seemed to me that the goal of

openhands/

provider was to have a curated list of LLMs that work best. If not, what should it contain?

✅ Done! Here's my review of #5102 — short version: it's solid, ship it. I read the full diff and verified the lists against the branch, the test file, and the downstream consumers. What's good • The rule is enforced in the right place: the module docstring + a path-scoped

AGENTS.md

in

llm/utils/

, so the sweep automation (and any future human/agent) will actually hit it next time they touch these files. That directly answers the "make it remember so it doesn't ask again" concern. • Lists are internally consistent with the "two latest versions per line" rule, and every

VERIFIED_OPENHANDS_MODELS

entry still resolves to a provider list (the

trinity-large-thinking

exception is preserved). • Test updates are coherent, not just name-migrations: the

organize

fixture swaps `o3`/`gpt-4o` for still-verified models (if left as dropped models they'd leak into

"other"

and break the assertions), and the opus→haiku dated-alias test preserves the original intent. • PR description is thorough — before/after table, judgment calls flagged explicitly, out-of-scope files named, and the

qwen3-6-plus

→

qwen3.6-plus

rename verified against the actual proxy

/v1/models

. CI is green. Non-blocking nits 1.

gpt-5.5

is still the default

model

in

LLM

config and several

auth/openai.py

signatures/docstrings (e.g.

model: str = "gpt-5.5"

), so the "out-of-the-box" example model is no longer in the curated set. Not a breakage (that default goes through direct-OpenAI, not the allowlist), but worth a follow-up to bump it to

gpt-5.6

for consistency. 2.

kimi-for-coding

is listed under Moonshot but (unlike

deepseek-chat

) is not in

VERIFIED_OPENHANDS_MODELS

. Likely intentional (proxy doesn't serve the alias), just wanted to confirm it's not an accidental omission. 3.

test_verified_lists_keep_two_latest_versions_per_line

is a spot-check, not exhaustive — fine given the AGENTS.md is the real guard, but the rule will still rely on humans/agents actually following it when adding a new version. Verdict: approve. Nothing blocking; the gpt-5.5 default is the only thing I'd file as a quick follow-up.

@fresh-machine-37269 if your verdict is approve, then post it on the PR