Here’s a concise, complete audit of how model name patterns are used in agent-sdk today, what they change in llm.py, how this interacts with LiteLLM, what still requires our own patterns/policies, and how to refactor to simplify. 1. Model name canonicalization and matching • What ◦ normalize_model_name(model) (model_features.py) ▪︎ Lowercases, strips. ▪︎ If “/” present, uses basename after last “/”; drops Ollama-style “:tag”. ▪︎ If basename has a vendor prefix like “anthropic.”, drops it (only for specific vendor set). ▪︎ Drops “-gguf” suffix. ◦ model_matches(model, patterns) ▪︎ If pattern contains “/”, match against full raw model string (provider-qualified). ▪︎ Else match against normalized basename. • Implications ◦ “anthropic.claude-*” becomes “claude-*”: will match basename patterns. ◦ “anthropic/claude-*” becomes “claude-*”: will match basename patterns. ◦ “bedrock/us.anthropic.claude-*” basename is “us.anthropic.claude-*” (NOT stripped): will NOT match “claude-*” patterns. ◦ Practical gap: Bedrock region-qualified ids often bypass our basename patterns. 2. Function calling (patterns → behavior) • Patterns ◦ FUNCTION_CALLING_PATTERNS: families like claude-, gpt-4o, gpt-5*, o1-2024-12-17, o3*, o4-mini, gemini-2.5-pro, kimi, qwen3. • Where used ◦ llm.py::_init_model_info_and_caps(): _function_calling_active = (native_tool_calling if set) else features.supports_function_calling. ◦ NonNativeToolCallingMixin: if tools and not native FC → mock via prompt (convert messages). ◦ llm.py::completion(): passes tools to LiteLLM only when native FC is active; else uses mocked prompting. • Params added/removed ◦ Native FC ON: kwargs["tools"] present (and tool_choice respected). ◦ Native FC OFF: kwargs["tools"] removed; kwargs["tool_choice"] removed; may add kwargs["stop"] (see item 5). 3. Reasoning effort (patterns → behavior) • Patterns ◦ REASONING_EFFORT_PATTERNS: o1*, o3*, o4-mini*, gpt-5*, gemini-2.5-*. • Where used ◦ options/chat_options.py::select_chat_options(): ▪︎ Adds reasoning_effort if llm.reasoning_effort is set (defaults to “low” for gemini-2.5-pro if none). ▪︎ Removes temperature and top_p for these models (provider ignores them). • Params added/removed ◦ Adds reasoning_effort. ◦ Removes temperature, top_p. 4. Extended thinking (Anthropic 4.5) • Patterns ◦ EXTENDED_THINKING_PATTERNS: “claude-sonnet-4-5_”, “claude-haiku-4-5_”. • Where used ◦ options/chat_options.py::select_chat_options(): ▪︎ Adds thinking={"type":"enabled","budget_tokens": llm.extended_thinking_budget}. ▪︎ Adds extra_headers={"anthropic-beta":"interleaved-thinking-2025-05-14"}. ▪︎ Forces out["max_tokens"] = llm.max_output_tokens. ▪︎ Removes temperature and top_p. • Params added/removed ◦ Adds thinking + extra_headers; sets max_tokens; removes temp/top_p. 5. Stop words in mocked FC (denylist patterns) • Patterns ◦ SUPPORTS_STOP_WORDS_FALSE_PATTERNS: “o1_”, “grok-4-0709”, “grok-code-fast-1”, “deepseek-r1-0528_”. • Where used ◦ NonNativeToolCallingMixin::pre_request_prompt_mock(): ▪︎ If not denied and not disabled, adds kwargs["stop"] = STOP_WORDS during mocked FC. • Params added/removed ◦ Adds stop only in mocked FC and only if allowed. 6. Prompt caching (Anthropic) • Patterns ◦ PROMPT_CACHE_PATTERNS: various claude-* families. • Where used ◦ llm.py::is_caching_prompt_active(): True only if llm.caching_prompt and pattern matches. ◦ llm.py::format_messages_for_llm(): ▪︎ If active, _apply_prompt_caching() sets Anthropic-style cache breakpoints (system first chunk; last user/tool chunk). • Params added/removed ◦ No request param; modifies message content flags (Anthropic-specific behavior). 7. Responses API routing (policy patterns) • Patterns ◦ RESPONSES_API_PATTERNS: “gpt-5*”. • Where used ◦ llm.py::uses_responses_api(): returns True for gpt-5*. ◦ openhands.sdk.agent.Agent.step(): chooses llm.responses() when True, otherwise llm.completion(). • Params added/removed (responses_options.py) ◦ Forces temperature=1.0, tool_choice="auto". ◦ store defaults to False; when store=False, add "reasoning.encrypted_content" to include. ◦ Adds reasoning={"effort": llm.reasoning_effort or "high", "summary":"detailed"}. 8. Provider-specific tweaks (string checks, not in pattern table) • Where used ◦ llm.py::_coerce_inputs(): ▪︎ Azure: default api_version if none. ▪︎ huggingface*: if top_p==1.0 → set to 0.9. ▪︎ openhands/ → rewrite to litellm_proxy/ and set base_url. ▪︎ Default reasoning_effort: “high” unless model contains gemini-2.5-pro or claude-4.5 models. ◦ llm.py::format_messages_for_llm(): ▪︎ deepseek or kimi-k2-instruct on groq → message.force_string_serializer = True. 9. Anthropic max_output_tokens override • Where used ◦ llm.py::_init_model_info_and_caps(): ▪︎ If max_output_tokens unset AND model in a specific Anthropic family, sets max_output_tokens = 64000; else takes from _model_info if available. 10. Vision support detection (mixed sources) • Where used ◦ llm.py::_supports_vision(): ▪︎ uses supports_vision(model) OR supports_vision(basename) OR _model_info.supports_vision. ◦ Rationale: historical LiteLLM bug for prefixed names; fallback to model_info. 11. How llm.py gets and uses LiteLLM model_info • Sources (llm.py::_init_model_info_and_caps()) ◦ openrouter/* → get_model_info(self.model) ◦ litellm_proxy/* → GET {base_url}/v1/model/info then take that model’s “model_info” ◦ Fallback → get_model_info(self.model.split(":")[0]) or get_model_info(self.model.split("/")[-1]) • Uses ◦ Sets max_input_tokens and max_output_tokens if unset. ◦ Used as vision fallback (supports_vision from model_info). 12. How LiteLLM computes supports_* / model_info • LiteLLM logic ◦ get_llm_provider() canonicalizes provider+model (handles prefixes properly). ◦ _get_model_info_helper() returns model_info (costs, token limits, supports_* flags). ◦ supports_* APIs check model_info or provider overrides; robust to prefixed names. 13. What patterns/policies we still need in agent-sdk (beyond model_info) • Keep (SDK policy or not exposed by LiteLLM): ◦ Extended thinking (Anthropic 4.5): add thinking + Anthropic beta header. ◦ Responses API default policy (“gpt-5*” or config flag). ◦ Stop words denylist (for mocked FC). ◦ huggingface top_p quirk (set 0.9 when 1.0). ◦ Azure default api_version. ◦ Serializer toggles for deepseek and kimi+groq. ◦ openhands/ → litellm_proxy/ rewrite. ◦ Possibly Anthropic 64k override (prefer model_info if present). • Delegate to LiteLLM (remove our patterns): ◦ supports_function_calling → supports_function_calling() ◦ supports_prompt_cache → supports_prompt_caching() ◦ supports_reasoning → supports_reasoning() ◦ supports_vision → supports_vision() (keep fallback for now) 14. Gaps/risks in current code • Bedrock “us.anthropic.*” models fail basename matches; features erroneously off. • Duplication with LiteLLM (patterns drift over time). • Vision detection workaround may be obsolete when LiteLLM fixes prefixed-name support. • Anthropic max_output_tokens override may diverge from real caps. 15. Refactor alternatives • A) LiteLLM-first adapter ◦ Build a capabilities adapter that uses supports_* from LiteLLM, then layer only SDK-specific overrides (extended thinking, stop denylist, responses policy). • B) Explicit policy + supports_* ◦ Keep a small file for our policy defaults (Responses, reasoning default, HF/azure quirks), delegate capabilities to supports_*. • C) Provider-qualified-only patterns ◦ If patterns remain, require “/”-qualified matches to avoid basename ambiguity (still inferior to supports_*). 16. Recommended plan (phased) • Phase 1: Add a get_capabilities() that: ◦ Canonicalizes model/provider via get_llm_provider(). ◦ Resolves supports_* via LiteLLM. ◦ Adds SDK-only overrides (extended thinking, stop denylist, responses policy). • Phase 2: Replace get_features() usage with the adapter internally; drop FUNCTION_CALLING, PROMPT_CACHE, REASONING_EFFORT patterns. • Phase 3: Make prefer_responses_api configurable (auto|true|false); keep HF/azure quirks and serializer toggles; constrain 64k override or remove if model_info provides caps. • Phase 4: Tests for anthropic/claude across: anthropic., anthropic/, bedrock/us.anthropic., openrouter/, litellm_proxy/; document the decision flow. Direct answer to the core question • Which model names/patterns do we still need in agent-sdk (beyond LiteLLM model_info)? ◦ Only those implementing SDK policy or features not present in LiteLLM’s model_info: ▪︎ Extended thinking (Anthropic 4.5) → adds thinking + anthropic-beta header. ▪︎ Responses API default routing (currently gpt-5*; could become config). ▪︎ Stop words denylist for mocked tool-calling (o1*, grok*, deepseek-r1). ▪︎ Provider quirks (HF top_p=0.9, Azure api_version default). ▪︎ Serializer toggles for deepseek/kimi-on-groq. ▪︎ openhands/ routing to litellm_proxy/. ▪︎ Optional Anthropic 64k cap override (prefer model_info when available). • Everything else (function-calling, reasoning capability, prompt caching, and vision) should rely on LiteLLM supports_* and model_info instead of our long-lived pattern tables. This reduces drift, fixes bedrock/us.anthropic.*, and simplifies the code. No code changes were made in this analysis. If you want, I can implement the “LiteLLM-first capabilities adapter” and replace the pattern tables in a branch, then open a PR. Would you like me to proceed with that, or keep the plan as documentation for now? _{View full conversation}

Summary of analysis (no code changes made) 1. Model normalization and pattern matching • normalize_model_name strips provider prefixes like “anthropic.” and basenames after “/”, drops “:tag” and “-gguf”. • model_matches: ◦ Provider-qualified patterns (with “/”) match raw model string. ◦ Basename patterns match normalized name. • Implication: bedrock/us.anthropic.claude- *normalizes to “us.anthropic.claude-*” and won’t match “claude-*” basename patterns. This is a key mismatch today. 2. Native tool calling (allowlist policy) • Patterns: FUNCTION_CALLING_PATTERNS. • Effect: ◦ If tools present and native FC allowed (by allowlist or explicit override), we send tools to the model. ◦ Otherwise we mock tool-calling via prompt (NonNativeToolCallingMixin) and remove tools/tool_choice; may add stop sequences. • Rationale: This is not just a capability detector. It encodes a default policy to only enable native tool use for families that perform well, even if other models “support” FC per LiteLLM. Keep this allowlist. 3. Prompt caching (Anthropic-specific) • Patterns: PROMPT_CACHE_PATTERNS. • Effect: When enabled, we annotate messages with Anthropic-style cache breakpoints (first system chunk; last user/tool chunk). • Rationale: Other providers may 400 on these flags. Keep Anthropic-specific gating. 4. Reasoning and extended thinking (distinct behaviors) • Patterns: REASONING_EFFORT_PATTERNS and EXTENDED_THINKING_PATTERNS. • Effect: ◦ Reasoning-capable families: add reasoning_effort and drop temperature/top_p. ◦ Anthropic 4.5 extended thinking: add thinking={type: enabled, budget_tokens}, set Anthropic beta header, force max_tokens, and drop temperature/top_p. • Rationale: Extended thinking is bespoke (not generic “reasoning”). Keep this separate logic/allowlist. 5. Stop words with mocked tool-calling (denylist) • Patterns: SUPPORTS_STOP_WORDS_FALSE_PATTERNS. • Effect: When mocking FC, add stop only if not denied. Prevents breakage on models that mishandle stop. • Rationale: Keep denylist. 6. Responses API default routing • Patterns: RESPONSES_API_PATTERNS (gpt-5*). • Effect: Those models use llm.responses() path with SDK defaults (temperature=1.0, tool_choice=auto, include encrypted reasoning when store=False, set reasoning summary). • Rationale: This is an SDK policy. Keep allowlist or make configurable. 7. Vision support • Detection: uses LiteLLM supports_vision(model) and basename fallback, with model_info fallback. • Rationale: Current approach is fine; if we want to force-enable for unknown models, consider an explicit override flag (not required now). 8. Provider quirks and routing • Azure: default api_version for azure*. • HuggingFace: if top_p==1.0, set to 0.9. • Routing: openhands/ → litellm_proxy/ with base_url set. • Serialization toggles: deepseek and kimi-on-groq force string serializer. • Anthropic max_output_tokens override: 64k for specific families when model_info doesn’t provide it. • Rationale: Keep these product/provider-specific behaviors. 9. How we use LiteLLM model_info/supports_* • llm.py loads model_info via openrouter path, litellm_proxy, or fallback to split forms. • Uses model_info for token limits and as a vision fallback. • We currently use LiteLLM supports_vision(), but not supports_function_calling/prompt_caching/reasoning for defaults. 10. What to keep vs delegate to LiteLLM • Keep as SDK policy/allowlists: ◦ Default native tool calling allowlist (performance-driven). ◦ Anthropic prompt-caching breakpoint gating. ◦ Stop words denylist for mocked FC. ◦ Responses API default routing (gpt-5* or config). ◦ Extended thinking allowlist + Anthropic-specific parameters. ◦ Provider quirks (Azure version, HF top_p, serializer toggles, routing, Anthropic 64k override). • Delegate to LiteLLM for capability detection (without flipping defaults): ◦ supports_function_calling, supports_prompt_caching, supports_reasoning, supports_vision. • Improve robustness: ◦ Canonicalize models via litellm.get_llm_provider before applying our allowlists to fix bedrock/us.anthropic.* mismatches. 11. Refactor plan (preserving behavior) • Add capabilities adapter: ◦ Canonicalize provider/model via get_llm_provider. ◦ Read supports_* and model_info from LiteLLM for “can this model support X?” signals. • Add policy adapter: ◦ Maintain curated defaults: enable_native_fc, apply_anthropic_cache_breakpoints, responses_api_default, stop_words_allowed_in_mocked_fc. • Wire-up: ◦ Continue to use our allowlists to decide defaults; use supports_* only to inform. ◦ Keep existing provider quirks and routing; optionally constrain Anthropic 64k override to Anthropic if model_info missing. • Tests: include anthropic., anthropic/, bedrock/us.anthropic., openrouter/, litellm_proxy/ variants to ensure correct decisions and avoid 400s on non-Anthropic. Corrections to my earlier draft • We should not drop the native tool calling allowlist nor the Anthropic prompt cache gating. These are behavior/policy choices and provider-specific mechanics, not just capability detection. • Extended thinking is separate from generic “reasoning”; keep bespoke parameters and header. • Vision is fine as-is; if needed, add an explicit override rather than patterns. No code changes were made in this analysis. If you want, I can implement the capabilities+policy split with LiteLLM canonicalization (preserving current behavior) and open a PR. How would you like me to proceed: • Implement on a new branch and create a PR, • Keep the changes local for now, or • Follow a different git workflow? _{View full conversation}