Hi, I can not seem to get tool calling to work wit...
# questions
b
Hi, I can not seem to get tool calling to work with OpenHands Cli 1.7.0 and gpt-oss-120b running on vLLM 0.11.0 (it is working just fine with Qwen32b). After I accept a tool use in the cli I get an error like this one:
Error: Expected markup value (found '=[{\'text\': "Here\'s the fil...them.", \'type\': \'text\'}], input_type=list]\n').
I am serving gpt-oss with the following vLLM command:
vllm serve openai/gpt-oss-120b --tool-call-parser openai --enable-auto-tool-choice --host <omitted> --port <omitted>
This happens every time a tool is called.
l
I haven’t encountered this error before, but it looks like it might be an issue on the VLLM side rather than the OpenHands side? Can you check if you’re able to get tool calls to parse when just directly calling the API for the vllm-served endpoint?
c
Copy code
found '=[{\'text\': "Here\'s the fil...them.", \'type\': \'text\'}]
this seems weird (the leading
=[{
and feels like a LLM serving issue?
can you try later version of vllm? irrc i had run into a bunch of bugs when try to serve gpt-oss using vllm
b
I tried the LiteLLM python example and although it did run without errors the output was a bit different than the example (only the first tool was called and the LLM reasoned whether to run the following ones). I will try Xingyao's suggestion and upgrade vLLm and if that fails evaluate another model. Thank you!
👍 1