https://livekit.io logo
Join Slack
Powered by
how do i call a specific function tool in llm_node...
# ask-ai
a
how do i call a specific function tool in llm_node without processing the response
t
Thanks for reaching out, Harshita. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Hey there! If you want to call a specific function tool in 
llm_node
without having to process the response, here’s a simple way to do it: 1. Define the function tool: Use the 
@function_tool
decorator to set up and register your function in the system. For example:
Copy code
from livekit.agents.llm import function_tool

@function_tool(name="my_function")
async def my_function(context: RunContext):
    try:
        # Your function implementation here
        print("Function executed successfully")
        return None, "Function executed successfully"
    finally:
        raise StopResponse()
1. Register the function: Just add it to the 
ToolContext
like this:
Copy code
from livekit.agents.llm.tool_context import ToolContext

tool_context = ToolContext([my_function])
1. Call the function: When you want to call it, include the function name in the chat context:
Copy code
await session.generate_reply([
    ChatContext().append({
        "role": "user",
        "content": "Call my_function"
    })
])
1. Prevent response processing: Inside your function, use 
StopResponse()
to stop any further response processing right after it runs. Doing this makes sure your function runs smoothly without the LLM generating any extra response. Hope this helps!
a
how do i use it in llm_node
t
Hey there! If you want to use a function tool in 
llm_node
, here’s a simple step-by-step guide to help you out: 1. Define Your Function: Start by creating a Python function and decorate it with 
@function_tool
. Here’s a quick example:
Copy code
from livekit.agents.llm import function_tool

@function_tool
async def my_function_name(parameters):
    # Your function code goes here
    pass
1. Add the Function to the Agent: Next, include that decorated function in your agent’s tools so it’s ready to use:
Copy code
from livekit.agents import Agent

class MyAgent(Agent):
    @function_tool
    async def my_function_name(self, parameters):
        # Your implementation here

    async def on_enter(self):
        self.session.generate_reply()
1. Invoke the Tool: The LLM can now call your function by name during interactions. For example:
Copy code
async def entrypoint(ctx: JobContext):
    session = AgentSession(
        agent=MyAgent(),
        room=ctx.room,
    )

    await session.start()
1. Add Tools Dynamically: Want to add tools on the fly? Use 
agent.update_tools()
like this:
Copy code
async def entrypoint(ctx: JobContext):
    agent = MyAgent()
    await agent.update_tools(agent.tools + [new_tool])

    session = AgentSession(
        agent=agent,
        room=ctx.room,
    )
1. Handle Errors: If something goes wrong, you can catch exceptions using 
ToolError
to keep things smooth:
Copy code
from livekit.agents.llm import ToolError

@function_tool
async def my_function():
    try:
        # Your code here
    except Exception as e:
        raise ToolError(f"Oops, something went wrong: {e}")
Follow these steps, and you’ll be all set to integrate and use function tools in 
llm_node
like a pro. If you have any questions, just ask—I’m here to help!
could you describe what you are trying to do? it sounds like you want to call a tool function from inside of your 
llm_node
implementation. Is that right? If so you can do it like this:
Copy code
async def llm_node(
            self,
            chat_ctx: llm.ChatContext,
            tools: list[llm.FunctionTool],
            model_settings: ModelSettings
    ):

        chat_ctx.add_message(
            role="system",
            content="call function my_function")
        return Agent.default.llm_node(self, chat_ctx, tools, model_settings)
you mentioned "without processing the response". What do you mean by that?
👍 1
a
how to control llm ttft
4 Views
Open in Slack
PreviousNext
Powered by