Can I access to the room from the LLM step? I have...
# ask-ai
b
Can I access to the room from the LLM step? I have the chat_context, but it does not seem to have access to the room. Using the agent library I want to trigger an RPC method there.
t
Thanks for reaching out, Martín. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Yes, you can access the room from the LLM step using the agent library to trigger an RPC method. The room is accessible through the JobContext (
ctx.room
), and from there you can perform RPC calls using the local participant. Here’s an example of how to access the room and perform an RPC call:
Copy code
# Access the room from the context
room = userdata.ctx.room

# Get remote participants to send RPC to
remote_participants = list(room.remote_participants.values())
if remote_participants:
    client_participant = remote_participants[0]
    
    # Perform RPC call
    await room.local_participant.perform_rpc(
        destination_identity=client_participant.identity,
        method="client.navigate",
        payload=json.dumps(payload)
    )
This approach is particularly useful when forwarding function calls from an LLM to the frontend. For example, in a function tool implementation:
Copy code
from livekit.agents import llm

class AssistantFnc(llm.FunctionContext):
    @llm.ai_callable()
    async def get_user_location(self, high_accuracy: bool = False):
        """Retrieve the user's current geolocation as lat/lng."""
        try:
            return await ctx.room.local_participant.perform_rpc(
                destination_identity=participant.identity,
                method="getUserLocation",
                payload=json.dumps({
                    "highAccuracy": high_accuracy
                }),
                response_timeout=10.0 if high_accuracy else 5.0,
            )
        except Exception:
            return "Unable to retrieve user location"
You can also register RPC methods that the agent will respond to:
Copy code
@ctx.room.local_participant.register_rpc_method("start_turn")
async def start_turn(data: rtc.RpcInvocationData):
    # Handle the RPC call
    session.interrupt()
    session.clear_user_turn()
The RPC system allows your LLM to directly access data or manipulate UI in your application’s frontend by performing method calls on remote participants within the room. Sources: complex-agents/turn-taking/agent.py | Function calling with Voice Agents | LiveKit Docs | examples/voice_agents/push_to_talk.py | Remote method calls (RPC) | LiveKit Docs
b
I have an instance of LLM: class MyLLM(LLM): … How can I do it there?
t
Hey there! Let me break down how you can work with rooms and RPC methods in your custom LLM class. Think of it like setting up a friendly communication system between your code and other participants. 😊
Copy code
class MyLLM(LLM):
    def __init__(self, room):
        self.room = room
        self._register_rpc_methods()

    def _register_rpc_methods(self):
        # Here's where we set up our communication channels
        self.room.local_participant.register_rpc_method(
            "agent.flipFlashCard",
            self.handle_flip_flash_card
        )

    async def handle_flip_flash_card(self, rpc_data):
        # We'll handle this RPC call with a friendly response
        return "Card flipped successfully"

    async def trigger_client_method(self, client_identity, payload):
        # And here's how we can reach out to other clients
        await self.room.local_participant.perform_rpc(
            destination_identity=client_identity,
            method="client.eou_update",
            payload=json.dumps(payload)
        )
### Here's the Scoop: • When you create your LLM class, just pass in the
room
object - it's like introducing a new friend to the group! • Use
room.local_participant.register_rpc_method
to set up methods that other participants can call. It's like creating open communication channels. • And with
room.local_participant.perform_rpc
, you can easily reach out and call methods on other participants. This approach lets your custom LLM class be super flexible - you can both listen for and initiate conversations with different parts of your application. Pretty cool, right? 🚀 Sources: examples/rpc/rpc-demo.ts | examples/rpc/index.ts | examples/rpc.py | complex-agents/turn-taking/agent.py
b
But that’s defining the RPC, I want to invoke it
t
Hey there! Let me walk you through how to call an RPC method from your custom LLM class. It's actually pretty straightforward – think of it like sending a quick message to a friend in the room. 😊
Copy code
class MyLLM(LLM):
    def __init__(self, room):
        self.room = room

    async def invoke_rpc(self, destination_identity, method, payload):
        try:
            response = await self.room.local_participant.perform_rpc(
                destination_identity=destination_identity,
                method=method,
                payload=json.dumps(payload)
            )
            print(f"RPC response: {response}")
            return response
        except Exception as e:
            print(f"RPC call failed: {e}")
            raise
### Key Parameters: • destination_identity: Who you want to send the message to. • method: The specific action you want them to do (like saying "greet"). • payload: The actual message you're sending (usually in JSON). ### Example Usage:
Copy code
my_llm = MyLLM(room)
await my_llm.invoke_rpc(
    destination_identity="participant-identity",
    method="greet",
    payload={"message": "Hello from RPC!"}
)
### Quick Tips: • Make sure the destination participant has the method you're trying to call registered first. • Keep your message under 15KiB – we don't want any data overflow! • By default, the system will wait 10 seconds for a response, but you can adjust this if needed. And there you have it! This method lets you easily communicate and trigger actions with other participants in the room. Pretty cool, right? 🚀 Sources: RPC | LiveKit Docs | Function calling with Voice Agents | LiveKit Docs | Runtime/Scripts/Rpc.cs | Remote method calls (RPC) | LiveKit Docs | examples/rpc/index.ts