https://docs.all-hands.dev/ logo
Join Slack
Powered by
Hello everyone:raised_hands: I'm Seoyoung, an unde...
# feedback
j
Hello everyone🙌 I'm Seoyoung, an undergraduate researcher focusing on operating systems at a university in South Korea. I'm reaching out because I'm running into a timeout issue with local LLM on OpenHands and could really use some expert advice. My Setup & Situation - I am running OpenHands using a local LLM via Ollama. - Installed via uv. - I submitted a prompt, and OpenHands was actively working on it for almost 4 hours. However, it eventually failed to finish the request, throwing this error: litellm.Timeout: APITimeoutError-Request timed out. Error_str:Request timed out What I've investigated so far - openhands/core/config/llm_config.py:74 
timeout: int | None = Field(default=None)
- openhands/llm/llm.py:230 
timeout=self.config.timeout
It seems like if the default is None, then timeout=None is passed to the LiteLLM call, which theoretically means there shouldn't be a timeout. My Questions 1. Given that timeout=None is being passed, why would LiteLLM throw a timeout error after 4 hours? Is there an underlying default timeout? 2. Do you know any sections in the codebase where I can look deeper into how this request lifecycle is managed or where this 4-hour limit might be coming from?
m
Hey @jolly-cpu-13395 When 
timeout=None
is passed to LiteLLM, it does NOT mean "no timeout", it means "use LiteLLM's default timeout". LiteLLM's Default Timeout Values is 6000 seconds according to the official documentation: ( LiteLLM Config Settings Documentation) Regarding the second question: what are you using specifically? OpenHands V0, OpenHands V1, or the SDK?
j
Thank you so much for your help! I also checked the 6000-second timeout you mentioned. However, I am still a bit puzzled because, in my case, the prompt continued running for almost four hours before the timeout error finally occurred. I am wondering whether LiteLLM’s default timeout is really the direct cause here, or whether there might be another timeout layer somewhere in the stack. Also I am running the V1 web app using locally downloaded OpenHands code through the CLI, and the actual core logic is running inside a Docker container based on the SDK.
m
The OpenHands SDK has built-in retry logic with exponential backoff. The SDK defaults are: • 
num_retries = 5
• 
retry_multiplier = 8.0
• 
retry_min_wait = 8s
• 
retry_max_wait = 64s
• 
timeout = 300s
(5 minutes per request) With 5 retries, each potentially waiting up to the timeout + backoff delay, the total time can accumulate. However, this alone doesn't fully explain 4 hours... I'm not an Ollama expert, but was the request actually processing and not stuck? For local LLMs with Ollama, especially with large models or complex prompts, the inference itself can take hours. The "timeout" error might have occurred when: • The connection was dropped after a long inference • Ollama unloaded the model mid-generation (
OLLAMA_KEEP_ALIVE
default is only 5 minutes!) • A TCP keepalive or HTTP connection timeout at the OS/container level kicked in You could try with
Copy code
# OpenHands SDK timeout
export LLM_TIMEOUT=36000  # 10 hours

# Ollama keep model loaded indefinitely
export OLLAMA_KEEP_ALIVE=-1
export OLLAMA_LOAD_TIMEOUT=30m
• Could you share the exact error message/stack trace*? • Which Ollama model are you using (size/parameters)? • Was this a multi-step task that might involve multiple LLM calls?
j
Thank you for the follow-up questions. 1. Exact error message / stack trace The error shown in the UI was: 
litellm.Timeout: APITimeoutError - Request timed out
Unfortunately, I do not currently have the full stack trace captured, only the final error message displayed in the interface. 2. Ollama model I was using 
qwen3-coder:30b
through Ollama. According to 
ollama list
, the model is about 30.5B parameters and the stored model size is about 18.5 GB. During runtime, 
ollama ps
showed about 22 GB of memory usage. 3. Whether it was a multi-step task Yes, it was a multi-step task. The prompt asked OpenHands to create multiple Python source files and supporting files, execute the program, inspect generated outputs, and run tests. From what I observed, the task progressed to about 70% completion before timing out. It appears to have stopped around step 8, where it was supposed to write at least five tests in 
test_main.py
. So it did not reach the later stages such as actually running the tests and finishing the final summary. • Prompt used
Copy code
Create a new Python project in the current working directory that satisfies the following requirements.

Project name: task_pipeline_system

Requirements:
1. Implement it by splitting the code across multiple Python files.
2. Create at least the following files:
 - main.py
 - config.py
 - utils.py
 - task_queue.py
 - worker.py
 - storage.py
 - logger.py
 - validator.py
 - sample_tasks.json
 - test_main.py
3. Program functionality:
 - Read a list of tasks from sample_tasks.json
 - Each task should have id, type, payload, and priority
 - Validate the input format in validator.py
 - Sort tasks by priority in task_queue.py
 - Process each task differently in worker.py depending on the task type
 - Save the results to results.json in storage.py
 - Save execution logs to app.log in logger.py
4. Implement at least three task types:
 - sum_numbers
 - reverse_text
 - count_vowels
5. Put common helper functions in utils.py.
6. Put file paths and constants in config.py.
7. main.py should control the overall workflow.
8. Write at least five tests in test_main.py.
9. After execution, the program should actually process sample_tasks.json and generate results.json.
10. Also run the tests.
11. If an error occurs during execution, fix the cause and run it again.
12. Finally, show the list of generated files, a summary of the execution results, and the test results.

Additional conditions:
- Do not put everything into a single file; the code must be separated properly.
- Each file should not be trivially short.
- Actually go through the sequence: create files -> execute -> test -> fix if needed.
- Check intermediate results and complete the task to the end.
m
@jolly-cpu-13395 Thanks for the info The 4-hour limit probably isn't a single timeout. What likely happened is that several steps succeeded over nearly four hours. By the final step, the context window became so large, due to the accumulated history and generated code, that the LLM call triggered a timeout, causing the entire session to fail.
j
I will definitely go ahead and apply the environment variables you suggested and test it out on my end. I'll do some more exploring and experimenting based on your pointers. Thank you again for your help!
2 Views
Open in Slack
PreviousNext
Powered by