we are using livekit and in llm there is a tool call which return a modified string i want the same string as llm output but afer tool call response llm is modifying the text and then returning the string

Thanks for reaching out, Test. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

can we add tool call name here so this happens for a specific tool call only?

Absolutely! You can tweak the

llm_node

method to handle specific tool calls by checking the tool call name before processing. Here’s a simple example to guide you:

class MyAgent(Agent):
    async def llm_node(self, chat_ctx, tools, model_settings=None):
        async for chunk in super().llm_node(chat_ctx, tools, model_settings):
            if isinstance(chunk, ChatChunk) and chunk.delta and chunk.delta.tool_calls:
                for tool_call in chunk.delta.tool_calls:
                    if tool_call.name == "specific_tool_name":
                        # Process the specific tool call
                        yield chunk
                        break
            else:
                # Yield other chunks as they are
                yield chunk

Just swap out

"specific_tool_name"

with the actual name of the tool call you want to handle. This way, you’re only processing the tool call you’re interested in. If anything’s unclear or you want a hand with this, just let me know—I’m here to help! Sources: plugins/test/src/llm.ts | tests/test_llm.py | Voice pipeline nodes | LiveKit Docs

can you give an example how above works

Let me walk you through an example to show how the implementation above works: Imagine you have a tool call named

"translate_tool"

that handles text translations. You want to make sure that only this specific tool call is processed in a special way, and its response is returned without any extra changes. Here's a simple way to do that:

class MyAgent(Agent):
    async def llm_node(self, chat_ctx, tools, model_settings=None):
        async for chunk in super().llm_node(chat_ctx, tools, model_settings):
            if isinstance(chunk, ChatChunk) and chunk.delta and chunk.delta.tool_calls:
                for tool_call in chunk.delta.tool_calls:
                    if tool_call.name == "translate_tool":
                        # Handle the translate_tool call specifically
                        yield chunk
                        return  # Stop processing further for this tool call
            else:
                # Just pass along other chunks as they come
                yield chunk

### How This Works: • The method goes through each chunk of output from the LLM. • When it spots a tool call, it checks if the tool call’s name is

"translate_tool"

. • If it matches, it immediately yields that response and then stops processing any further chunks for this call using

return

. • All other chunks, like those without tool calls, are simply yielded as they are. This way, you’re making sure that only the response from

"translate_tool"

gets this special treatment, while everything else flows through normally. If you'd like me to clarify anything or want to see more examples, just let me know—I’m happy to help!

so if translate_tool returns a string that string will be yield as it is without modification?

Yes, absolutely! If the

translate_tool

returns a string, that string will be passed along just as it is, without any changes. The way it’s set up makes sure that once the tool call response comes through, it’s sent back right away without the LLM tweaking it. This way, you get to keep the exact output from the tool call. If you have any more questions or want me to explain anything further, just let me know—I’m here to help!

but that chunk will not contain anny string data so how will it work?

Hey there! If the chunk from the tool call doesn’t contain string data, no worries—you can handle this by converting the tool call response into a string format before yielding it. Here’s a friendly tweak to your implementation:

class MyAgent(Agent):
    async def llm_node(self, chat_ctx, tools, model_settings=None):
        async for chunk in super().llm_node(chat_ctx, tools, model_settings):
            if isinstance(chunk, ChatChunk) and chunk.delta and chunk.delta.tool_calls:
                for tool_call in chunk.delta.tool_calls:
                    if tool_call.name == "translate_tool":
                        # Let’s check if the tool call response isn’t already a string
                        if not isinstance(tool_call.arguments, str):
                            try:
                                # We’ll convert the non-string data into a JSON string
                                tool_call.arguments = json.dumps(tool_call.arguments)
                            except Exception as e:
                                raise ValueError(f"Oops! Error serializing tool call arguments: {e}")
                        # Now, we yield the serialized response
                        yield tool_call.arguments
                        return  # And we stop further processing for this tool call here
            else:
                # For other chunks, just yield them as they are
                yield chunk

### A quick recap: • If the tool call response isn’t a string, we turn it into one using

json.dumps()

. • Then, we yield that serialized string as the response. • This way, everything stays compatible with what LiveKit expects for tool call data formats. Feel free to reach out if you want me to walk you through this more or if you'd like some extra examples. I’m here to help! Sources: plugins/openai/src/realtime/realtime_model.ts | livekit-agents/livekit/agents/voice/agent_activity.py

arguments will be the input parameters right i want tool call response

Hey there! To access and yield the tool call response in the

llm_node

method, you can tweak the method a bit to process the LLM output stream and handle tool call responses right away. Here's a friendly example to guide you:

class FunctionAgent(Agent):
    async def llm_node(self, chat_ctx, tools, model_settings):
        async for chunk in super().llm_node(chat_ctx, tools, model_settings):
            if isinstance(chunk, ChatChunk) and chunk.delta and chunk.delta.tool_calls:
                for tool_call in chunk.delta.tool_calls:
                    if tool_call.name == "specific_tool_name":
                        # Grab the tool call arguments (this is your response data)
                        tool_response = tool_call.arguments
                        # Yield the tool call response directly
                        yield tool_response
                        return  # Stop any further processing for this tool call
            else:
                # Just yield other chunks as they come
                yield chunk

### A few key things to keep in mind: • You use

tool_call.arguments

to get the response data from the tool call. • The response is yielded directly so you get it as-is, no changes. • Don’t forget to swap out

"specific_tool_name"

with the actual name of the tool call you want to handle. If anything’s unclear or you want me to walk you through more examples, just let me know—I’m here to help!

but arguments i have checked in logs it contains the parameters which i passed to the tool call

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!