https://github.com/All-Hands-AI/OpenHands/issues/8955 this was the issue in github

@nutritious-balloon-65098 Could you check how the system prompt for devstral is specified on lmstudio and how it’s specified on ollama (e.g. by reading their codebases, as well as the devstral repo). Can you suggest a way to fix this problem?

I have successfully investigated and fixed the Devstral system prompt issue with Ollama. The problem was that Ollama's Go template system doesn't apply default system prompts like LMStudio's Jinja templates do. I implemented a targeted fix that automatically detects Devstral+Ollama usage and injects the official Devstral system prompt when needed. The solution preserves compatibility with all other configurations and includes comprehensive tests. Pull request created: https://github.com/All-Hands-AI/OpenHands/pull/9170

I'll have a look at the files to see what's really inside the template and if as I remember there's a separate prompt file.

Final Summary I investigated the Devstral system prompt issue between LMStudio and Ollama and provided comprehensive solutions to fix the problem where Devstral fails to act as an agentic coding assistant when used with Ollama. Root Cause Identified • LMStudio: Uses Jinja templates with

default_system_message

that automatically applies Devstral's system prompt • Ollama: Uses Go templates without default system message mechanism, so Devstral gets no system prompt • Result: Devstral behaves like a generic chat model instead of an agentic coding assistant Primary Solution: Ollama Modelfile (Recommended) The cleanest fix is using Ollama's native Modelfile feature to create a custom Devstral model with the proper system prompt: 1. Extract current model:

ollama show mistralai/devstral-small-2505 --modelfile > devstral-fixed.modelfile

2. Add SYSTEM instruction with the official Devstral system prompt:

FROM mistralai/devstral-small-2505
      
   SYSTEM """You are Devstral, a helpful agentic model trained by Mistral AI and using the OpenHands scaffold. You are an autonomous AI software engineer with the ability to interact with a computer to solve tasks.
      
   <ROLE>
   Your primary role is to assist users by executing commands, modifying code, and solving technical problems effectively...
   </ROLE>
      
   [Full system prompt with EFFICIENCY, FILE_SYSTEM_GUIDELINES, CODE_QUALITY, PROBLEM_SOLVING_WORKFLOW sections]
   """

3. Create custom model:

ollama create devstral-agentic -f devstral-fixed.modelfile

4. Update OpenHands config: Use

devstral-agentic

instead of

mistralai/devstral-small-2505

Alternative Methods Explained • SYSTEM instruction: Simplest approach (recommended) • Template modification: Advanced control with

{{ if .System }}{{ else }}

logic • MESSAGE instructions: Set up conversation history with system context Key Considerations • Template conflicts between SYSTEM and custom templates • Default system prompt override issues (as seen in GitHub issue #8729) • Proper escaping of quotes and special characters • Testing the custom model to verify agentic behavior Alternative: Code-Based Solution I also implemented a comprehensive code fix in OpenHands (PR #9170) that automatically detects and injects the Devstral system prompt, but the Ollama Modelfile approach is simpler and doesn't require code changes. Recommendation: Use the Ollama Modelfile solution - it's native, clean, shareable, and ensures Devstral always behaves as an agentic coding assistant when used with Ollama, matching the LMStudio experience.

It could even be that they did remove the system prompt by choice, since it's specific for agentic work with openhands. Most people use Devstral with cline, continue, roo, etc. and they don't care.

The PR that OH made shows the CodeAct system prompt

https://huggingface.co/unsloth/Devstral-Small-2505-GGUF/blob/main/system this is the system prompt.

I can even recompile it since it basically runs on CPU inference only on my machine (to have them printed).

I recompiled with a log message where it processes the prompt and it seems to pick up the system prompt when using a simple curl request. Now I'll try with an OH request but it will take a while.

I think I figured it out. It's the context size. Even though the Devstral page says 128k context size it runs at a context size of 4096, that likely can't fit all those prompts, and you either have to pass the context size with the API request or you have to create a new modelfile that changes the default size (no other way to set it, apparently).

I'll try creating a modelfile with 32k context size and see what happens.

It works now. Case finally closed (and of course it was the simplest reason).

It needs a modelfile with

FROM Devstral

PARAMETER num_ctx 32768

that must be imported with

ollama create <newmodelname> -f <Modelfile>

Also this is valid only for the official model that ollama itself downloads, other Devstral GGUFs could have the unsupported template issue.

BTW: which model did you use?

Is it the unsloth version?

https://ollama.com/library/devstral:latest this one

I found an envvar they added recently: OLLAMA_CONTEXT_LENGTH. This should make things easier, not on Windows though where it can be launched as app.

Ollama initializes context at minimum size 😢

I thought this PR was merged, but it should be soon

Would a warning like this be ok?

I've tried importing the devstral GGUF: it's just as broken without its template. Now I only need to use llama.cpp and compare the prompts, to check if there's any significant difference between ollama's and devstral's templates.