Hey everyone! I'm running fedora and followed the prescribed local llm setup (with lmstudio + the recommended qwen model). Unfortunately, when I click "new conversation", I get a (not particularly detailed) connection error (

litellm.InternalServerError: InternalServerError: OpenAIException - Connection error.

) How do I debug this?

Could you tell which model exactly you wrote, and if it was pulled locally?

Yeah of course! It was this guy - Qwen3.6 35B A3B

In the openhands LLM settings, how do you write the model? Full name, with openai/ prefix? and base URL? I do ask exactly, because I think maybe it didn’t manage to connect at all

Yeah it may not have connected. Lemme get you the settings I passed, one sec...

ah, I assume you’re using openhands/openhands with docker

Yes. I used the exact command in the openhands local llm tutorial

How about curl with /v1 at the end?

curl http://127.0.0.1:1234/v1

{
  "error": "Unexpected endpoint or method. (GET /v1)"
}

The agent is inside a docker container, the LLM call has to get to your server which is on the host

Hmm so you think it could be a docker / networking issue?

yes, it’s my hypothesis 🤔 the second tiny thing you could try, is to use only openai/ prefix and not qwen/

Trying that. Looks like it's still hanging unfortunately

yeah those curl results are actually good news, that's the server replying not refusing you. it's almost certainly a docker networking thing on fedora, not lmstudio couple things to try: first, curl the right endpoint to see the actual model id:

curl <http://127.0.0.1:1234/v1/models>

that'll show you exactly what to put after openai/ then the real fix is probably two parts. on linux host.docker.internal doesn't resolve by default like it does on mac/windows, so the openhands container literally can't find your host. add this to the docker run command:

--add-host host.docker.internal:host-gateway

and lmstudio by default usually only binds to 127.0.0.1 which a container can't reach anyway. flip it to serve on 0.0.0.0 (there's a network/serve on local network toggle in the server settings) once both of those are sorted set the model to openai/ + whatever id the /v1/models call gave you. wouldn't drop the qwen/ bit unless the model id actually doesn't have it

Appreciate the help from both of you! Your curl command did return something!!

"data": [
    {
      "id": "qwen/qwen3.6-35b-a3b",
      "object": "model",
      "owned_by": "organization_owner"
    },
    {
      "id": "text-embedding-nomic-embed-text-v1.5",
      "object": "model",
      "owned_by": "organization_owner"
    }
  ],
  "object": "list"

Restarting openhands...

good, so the model id is right (qwen/qwen3.6-35b-a3b, so your openai/qwen/qwen3.6-35b-a3b is correct) and if --add-host was already there then the hostname resolution isn't your problem either. that points at lmstudio binding. can you check what address it's actually serving on? if it's still 127.0.0.1 the container reaches the host but nothing's listening on the interface it can see quickest way to confirm from inside the container, run:

docker exec -it <openhands-container-name> curl -s <http://host.docker.internal:1234/v1/models>

if that hangs or errors it's the bind. if it returns the model list then networking's fine and we look elsewhere

Going to investigate turning on 0.0.0.0 serving now...

docker ps

docker exec -it df4413aaa933 curl -s http://host.docker.internal:1234/v1/models returned nothing

that bit's actually in lmstudio, not docker. in the server/developer tab (where you start the local server) there's a "Serve on Local Network" toggle, turn that on. that's what switches it from 127.0.0.1 to 0.0.0.0 so it listens on all interfaces instead of just localhost you should see the address flip to 0.0.0.0 once it's on. then rerun the docker exec command and it should return the model list, and your new conversation in openhands should work

Tried running the lmstudio with a --bind 0.0.0.0 flag. Running curl gives this: curl http://0.0.0.0:1234/v1/models { "data": [ { "id": "qwen/qwen3.6-35b-a3b", "object": "model", "owned_by": "organization_owner" }, { "id": "text-embedding-nomic-embed-text-v1.5", "object": "model", "owned_by": "organization_owner" } ], "object": "list"

I don’t remember lmstudio interface atm; does it have this?

https://openhands-ai.slack.com/archives/C06U8UTKSAD/p1780776821712289?thread_ts=1780775260.704799&channel=C06U8UTKSAD&message_ts=1780776821.712289

OK WAIT I CAN TALK TO IT THANK YOU BOTH SO MUCH! Got a different error - litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': 'The number of tokens to keep from the initial prompt is greater than the context length (n_keep: 16215>= n_ctx: 4096). Try to load the model with a larger context length, or provide a shorter input.'}

Maybe 32k works if you can, or 64k better imho

Oh ok I'll try 32 / 64k then.

Glad it's working! Added a docs PR so future Linux users don't hit the same wall: https://github.com/OpenHands/docs/pull/555 Adds a warning to the LM Studio quickstart about enabling "Serve on Local Network" on Linux, plus a

docker exec curl

command to verify connectivity.

no probs Steve, glad you're up and running! sorry i dropped off, got stuck deep in a custom built AI gateway in go looks like you nailed it in the end, the local network toggle plus bumping context up to 32/64k. for what it's worth that context length thing will bite you again if you load a bigger model later, so worth remembering openhands needs a good chunk of room to work with. and good shout from smolpaws getting it into the docs, that linux 0.0.0.0 bind catches loads of people out shout if you hit anything else!