I get frustrated when I have the wrong API key for...
# proj-agent-canvas
a
I get frustrated when I have the wrong API key for a LLM and i have to wait to start and conversation or automation to find out it was wrong, i saw someone had put this feature request in - github.com/OpenHands/OpenHands/pull/18940#… does this fit with what we are doing for validating API keys right away
l
I added a PreFlight a while ago to make sure that you can't enter invalid API keys, is that not working?
@fresh-machine-37269 why is the preflight currently not working?
👀 1
f
🤖 On it! View progress here: oss-agent-canvas.ngrok.dev/conversations/…
✅ Done! Looked into it — the preflight check itself isn't gone, but its failure feedback is effectively invisible right now, which is why it looks broken: 1. The green check is misleading. The profile editor shows a ✓ as soon as the API key field is merely non-empty — it doesn't mean the key was validated. 2. Real failures only surface as a transient toast. When the save-time configuration check (the actual preflight against the LLM) fails — e.g. an invalid key — the error is reported via a temporary toast that's easy to miss, and nothing persistent stays on screen. 3. Stale results get discarded. If the draft changes while a preflight is in flight, the result is dropped, and late results don't reliably reflect what you're looking at — so users can save an invalid key thinking they just saw a pass. PR #18940 (
feat(settings): retain LLM profile validation failures
) fixes exactly this — it keeps a failed configuration check visible above the editor actions, maps SDK
LLMAuthenticationError
to credential-specific guidance, and discards obsolete preflight results properly. It's currently open and unmerged, stuck at the scope gate: the linked issue #18727 is still labeled
priority:low
without
ready-for-dev
, so it's waiting on a maintainer triage decision rather than being blocked on code. Given the frustration this causes, it may be worth bumping that priority / fast-tracking the merge.
m
i suppose we cannot have a preflight for every provider. But for litellm should be possible or?
l
We can, just send a small completion
m
ah ok… I was thinking without using money, because we can have false negative with this method. but I suppose for mostly of the cases is good.