https://docs.all-hands.dev/ logo
Join Slack
Powered by
Have you noticed that agent-server stops respondin...
# general
b
Have you noticed that agent-server stops responding all requests while it's waiting for the LLM request response?
👀 1
r
I had some stability trouble like that with the canvas backend. I thought it was just my hacky local setup and being new to openhands. But it was pretty consistent. And disheartening.
👍 1
b
It's a very old issue. You notice it more with local inference, especially while condensing.
r
Yeah im using rented RTX 5090s for inference and a local llm - qwen3.6 35B A3b.
b
careful with 35B, it destroys your codebase. Better use 27B
r
Howso? Just - not good coding ability? Im using it more for automation with tool-calling.
b
35B ignores instructions frequently
😦 1
smart, but has amnesia
and this is dangerous when you have a code base
for simple CLI commands, I guess it's fine
r
Thanks for the tip. I'll keep it in mind and watch its performance. I have no other reference points at the moment as im stepping into localllms from the frontier world as a backup. So I thought it was just because it was a non-frontier model.
👍 1
But in general 3.6 35B has been super solid for me and my case.
b
and super fast
🚀 1
the problem is the MoE architecture
it has some split-brain behaviour
t
was this a model issue? I seem to be having that same problem but i am using the qwen3.6-35B, from what i got in the conversation it is better to move to the 27B?
b
No, agent-server blocks waiting for the LLM response and other requests fail.
t
How did you rectify it? Some conversations are connecting, and my LLM was running properly, but it started running slowly this morning. I restarted the Docker containers, yet the agent-canvas still seems to be hanging with conversations. I also increased the LLM timeout.
b
you could be having other issues too
you could test if canceling the LLM request unblocks the agent-server
or simply check if it unblocks once the LLM request finishes
agent-server must be modified to not block requests while waiting for the LLM response
t
i see thank you
👍 1
4 Views
Open in Slack
PreviousNext
Powered by