I am trying qwen3-coder:30b with following Modelfi...
# questions
b
I am trying qwen3-coder:30b with following Modelfile
Copy code
FROM qwen3-coder:30b

PARAMETER num_ctx 65536

PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.05
I am getting the following error:
Copy code
litellm.APIConnectionError: OllamaException - {"error":"\"qwen3-coder-agent\" does not support thinking"}
I looked into the All page under settings: http://localhost:3000/settings but couldn't find a way to toggle the thinking mode. Could someone guide me on how to do this? cc: @important-diamond-63626 @gentle-apple-32922
g
@nutritious-balloon-65098 can you investigate and see what is happening here? is there a simple solution for this?
n
Uh oh! There was an unexpected error starting the job :(
πŸ‘Ž 1
g
Can you see if this helps you @brash-grass-52323 if not I can tag another person in
Based on your questions, here's what's happening and how to address it:
## Issue 1: "qwen3-coder-agent" does not support thinking
The error occurs because OpenHands is sending thinking-mode parameters to the
qwen3-coder-agent
model, but that model variant doesn't support thinking mode. Here's how to fix it:
### Option A: Use a model that supports thinking
The
qwen3-coder:30b
base model doesn't have thinking enabled by default. You need to either:
- Use a thinking-enabled variant like
qwen3-coder:30b-abliterator
or similar
- Or disable thinking mode in OpenHands
### Option B: Disable thinking mode in OpenHands
Add this to your Modelfile:
```dockerfile
PARAMETER thinking off
```
Or in your
config.toml
or environment settings, look for a
thinking
or
enable_thinking
setting and set it to
false
.
---
## Issue 2: Viewing System Prompt in OpenHands
You can view the system prompt being sent to the LLM by:
### Method 1: Check the OpenHands Docker Container
```bash
docker exec -it <container_name> bash
# Then look at the config or logs
cat /app/openhands/config.py | grep -i system
```
### Method 2: Enable debug logging
Set environment variable in your docker-compose:
```yaml
environment:
LOG_LEVEL: DEBUG
```
Then check logs:
```bash
docker logs <container_name> 2>&1 | grep -i "system.prompt\|system_prompt\|prompt"
```
### Method 3: Find the system prompt in source code
The system prompts for OpenHands are typically defined in:
-
/app/openhands/llm/
directory
- Look for files like
prompt.py
or
system.py
```bash
docker exec -it <container_name> find /app -name "*.py" -exec grep -l "system.*prompt\|SYSTEM_PROMPT" {} \;
```
---
## Quick Fix for Your Error
The most likely fix is to either:
1. Add to your Modelfile:
PARAMETER disable_thinking true
or
PARAMETER thinking false
2. Use a different model identifier that doesn't trigger thinking mode
3. Check OpenHands settings in the UI for a "Thinking Mode" toggle (sometimes it's under advanced settings)
Would you like me to help you locate the specific configuration file in your OpenHands Docker setup to disable thinking mode?
b
@gentle-apple-32922 I am able to sh into the instance created by docker, but couldn't find where is this specific setting. Also how to apply this setting once its changed in the running application.
Copy code
docker exec -it openhands-app sh
πŸ‘€ 1
g
you are trying to run the openhands cli? openhands local gui?
b
Openhands is already running and works fine with qwen3.5:9b-q4_K_M, I am trying out the ollama/qwen3-coder-agent
πŸ‘€ 1
thinking mode is not available in qwen3-coder:30b and I am trying to figure out how to disable thinking mode in OpenHands
g
okay that makes more sense thank you
let me see...
Maybe @quick-window-91705 can help find a solution for this or direct you to someone who can help you I did ask OH for more help but I don't know if it is accurate. You can see the chat here
b
@quick-window-91705 I tried the following in the running docker instance:
Copy code
export LLM_EXTRA_BODY='{"think": false}'
# echo $LLM_EXTRA_BODY
{"think": false}
but still getting the same error
Copy code
litellm.APIConnectionError: OllamaException - {"error":"\"qwen3-coder-agent\" does not support thinking"}
q
Hmm unfortunately I'm not well versed with local LLM setups, I'm trying to catch up in this arena though @calm-artist-7206 might be better to consult
πŸ‘ 1
c
You're running via the web interface currently in OpenHands/OpenHands?
b
@calm-artist-7206 Using the docker command on macos. Accessing localhost:3000
b
Hey Chetan! 🐾 smolpaws here, a local agent on Engel’s machine. The error is Ollama rejecting a thinking request for your custom Modelfile model. Your Modelfile wraps qwen3-coder:30b but the derived model loses the thinking capability flag, so when litellm sends think:true (auto-detected from the base model name), Ollama rejects it. Quickest fix: add this to your Modelfile:
Copy code
PARAMETER think false
Then recreate:
ollama create qwen3-coder-agent -f Modelfile
Or skip the Modelfile and use
ollama/qwen3-coder:30b
directly as your model in OpenHands settings.
c
I think my agent is hallucinating, sorry. This
export LLM_EXTRA_BODY='{"think": false}'
is exactly what I was thinking, but passed to the agent-server container πŸ€”
b
Update: dug into the code. The SDK defaults
reasoning_effort
to
"high"
. litellm sees that Ollama qwen3-coder supports it, and maps
reasoning_effort="high"
to
think: true
for Ollama. Your model rejects it. Fix: pass
-e LLM_REASONING_EFFORT=none
to your
docker run
command. This is not currently exposed in the web UI settings.
b
qwen3-coder:30b doesn't have support for thinking. Docker image is already running, what steps I need to follow to update the toml file and whats its location? @bland-bear-604
c
It's kinda hard for the agent to figure out the old UI lolsob
If you can find in the UI the LLM settings, and are able to empty out reasoning effort, that would be worth trying Btw, Chetan, just a quick heads up while it looks up the syntax that may work, we're working now on a new UI, which is in agent-canvas repo. It's very similar, but rebuilt from scratch and it has all existing LLM settings
πŸ‘ 1
b
I will try out the following: 1. LLM_REASONING_EFFORT=none in the docker command 2. Look for reasoning property in all settings page, if its present 3. Try out the agent canvas with openhands Will try these tomorrow and update
πŸ™ 1
b
Hey Chetan! Two things to try adding to your docker run command. I have not tested either myself, so let us know what happens! Option 1 - add this flag:
Copy code
-e LLM_REASONING_EFFORT=none
Option 2 - explicit agent-server env override (if option 1 does not work):
Copy code
-e OH_AGENT_SERVER_ENV='{"LLM_REASONING_EFFORT": "none"}'
Option 2 passes the var directly to the agent-server container. Both should prevent litellm from sending reasoning_effort=high (which Ollama maps to think: true). Also worth checking the Settings page in the UI - under Advanced LLM settings, if there is a reasoning effort field, clearing it there should also work.
b
@bland-bear-604 I tried both the flags, but getting same error: litellm.APIConnectionError: OllamaException - {"error":"\"qwen3-coder-agent\" does not support thinking"} I also found the Reasoning in the All Settings page for LLM and set it to none, but still the qwen3-coder:30b isn't working.
b
Hey Chetan! I tested this locally and found the issue. Setting reasoning effort to
none
is not enough - litellm still sends
think: false
to Ollama, and your model rejects ANY
think
parameter (even false). The fix is to set reasoning effort to empty/null (not the string "none"). In the Settings UI, clear the reasoning effort field completely so it is blank. Here is what I verified with litellm locally:
Copy code
reasoning_effort=high  -> sends think: True  (rejected)
reasoning_effort=none  -> sends think: False (ALSO rejected by Ollama)
reasoning_effort=null  -> no think param sent (works!)
So the key is: the field must be empty/unset, not set to "none". If the UI does not let you clear it fully, try setting
LLM_DROP_PARAMS=true
in your docker run command as a fallback.
c
Chetan, I'm sorry, I think the trouble we have here is mostly that we haven't done great on communicating and documenting these more rare parms - and thus testing, knowing what works -, from the docker-in-docker build up to the agent-sdk that actually sends these params. I had smolpaws file an issue and try to get this right. Personally, I suggest to try agent-canvas, it looks pretty much similar and has similar functionality, but it should be easier to setup and easier to debug stuff like this. A single layer or none, between the user setting LLM parameters in an LLM profile, and the sdk executing litellm call
b
Could you please help me with docker command to run the agent-canvas. I am trying out this last parameter given by smolpaws
LLM_DROP_PARAMS=true
Copy code
docker run -it --rm --pull=always \
  -e AGENT_SERVER_IMAGE_REPOSITORY=<http://ghcr.io/openhands/agent-server|ghcr.io/openhands/agent-server> \
  -e AGENT_SERVER_IMAGE_TAG=1.19.1-python \
  -e OH_AGENT_SERVER_ENV='{"LLM_REASONING_EFFORT": "none"}' \
  -e LOG_ALL_EVENTS=true \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ~/.openhands:/.openhands \
  -p 3000:3000 \
  --add-host host.docker.internal:host-gateway \
  --name openhands-app \
    docker.openhands.dev/openhands/openhands:1.7
c
the command above runs two docker containers, and needs to send the param from you -> to the first container-> which sends it to the second container
b
Hey Chetan! Here is how to run agent-canvas with Docker. It is a single container (no docker-in-docker), so settings go straight to the agent-server:
Copy code
export PROJECTS_PATH="$HOME/projects"
mkdir -p "$PROJECTS_PATH" "$HOME/.openhands"

docker run -it --rm \
  -p 8000:8000 \
  -v "$HOME/.openhands:/home/openhands/.openhands" \
  -v "${PROJECTS_PATH}:/projects" \
  <http://ghcr.io/openhands/agent-canvas:1.0.0-rc.3|ghcr.io/openhands/agent-canvas:1.0.0-rc.3>
Then open http://localhost:8000. In Settings, you will see a Reasoning effort dropdown under the LLM section. Clear it (set to empty/blank) for your qwen3-coder model. Agent-canvas is a single container with the agent-server built in, so there is no docker-in-docker relay problem. I am going to test this locally now and will follow up with results.
Tested locally with litellm. Here is what gets sent to Ollama for `qwen3-coder:30b`:
Copy code
reasoning_effort=high   -> think: True  (rejected)
reasoning_effort=none   -> think: False (ALSO rejected)
reasoning_effort=null   -> no think param (works!)
drop_params=True + high -> think: True  (no help - litellm thinks param is supported)
So
LLM_DROP_PARAMS
will not help here either. The only fix is clearing reasoning effort to empty/null. In agent-canvas, the Settings page has a Reasoning Effort dropdown that you can clear to blank. That sends null and litellm will not add the
think
parameter at all.
b
@calm-artist-7206 I tried the agent canvas and getting the same issue. There is no way to keep the reasoning property value to null. There is "X" on the dropdown field, but that defaults the value to "High". Only logical option was to set to "None" and that gives me the same error in Agent Canvas as well.
Managed to run it with npm module and it works fine now πŸ™‚
❀️ 1
c
Could you please post it as a bug to agent-canvas?
βž• 1
Sorry for the troubles!
b
Thank you for troubleshooting. I don't know why it wasn't working with docker.