hi

Thanks for reaching out, Rahib. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

1. Multiple AgentSessions in Same Room Can two separate AgentSession instances exist in the same LiveKit room simultaneously? For example, one voice agent that speaks to users, and one silent "observer" agent that only listens and monitors the conversation? 2. Creating Multiple Sessions in Single Entrypoint In a single Python worker entrypoint function, can I create and start two AgentSession instances that both connect to the same room? Or do I need to dispatch a separate agent worker? 3. RoomIO and Participant Targeting When using RoomIO with an AgentSession, can I use room_io.set_participant(identity) to subscribe to a specific participant's audio? Can I subscribe to BOTH a SIP participant (caller) AND another agent participant at the same time? 4. Cross-Session Communication If I have two agents in the same room (Agent A speaks, Agent B monitors), can Agent B call methods on Agent A's session? For example, can Agent B trigger agent_a_session.interrupt() or agent_a_session.generate_reply()? Or do I need to use RPC/data channels for cross-agent communication? 5. Muting Participants Programmatically How can an agent programmatically mute a SIP participant (caller) in a room? Is there a LiveKit API to mute/unmute a remote participant's audio track? I want to temporarily mute a caller, let the agent speak, then unmute them. 6. Silent Agent Pattern What's the recommended pattern for a "silent observer" agent that listens to audio but never speaks? Should I set tts=None in AgentSession, or use a dummy TTS like TTS(model="NO_TTS_USED")? Should I override the tts_node method to pass? 7. Agent Subscribing to Another Agent's Audio Can an agent subscribe to another agent's published audio track? In the call-moderation demo, the moderation agent subscribes to the driver agent's audio. How does this work with RoomIO and set_participant()? 8. Detecting Agent Turn Completion How can I detect when an agent has finished speaking its turn? Is there an event like agent_speech_completed or should I listen to agent_state_changed transitions (e.g., from "speaking" to "listening")? 9. Update Chat Context from External Source Can I call session.update_chat_ctx() from outside the agent's normal flow? For example, if an observer detects an issue, can it inject a system message into the main agent's context mid-conversation? 10. Agent Dispatch vs Same-Process Creation What's the difference between: - Creating a second AgentSession in the same entrypoint - Using api.agent_dispatch.create_dispatch() to spawn a separate agent worker When should I use each approach?

Thanks — I'll answer each of your 10 questions directly. 1) Multiple AgentSessions in Same Room Yes. Multiple AgentSession instances can join the same LiveKit room concurrently (e.g., a speaking agent plus a silent observer). Both join as AGENT participants and can coexist. 2) Creating Multiple Sessions in Single Entrypoint Yes. You can create and start multiple AgentSession instances in a single Python worker entrypoint and have them connect to the same room, as long as each session is managed (unique identities, lifecycle) within that process. 3) RoomIO and Participant Targeting You can use

room_io.set_participant(identity)

to target a specific participant’s audio. `RoomIO`’s simplified API is typically for one target at a time. To subscribe to both a SIP participant and another agent simultaneously, either run multiple sessions (each targeting one participant) or use the lower-level Room/RemoteParticipant subscription APIs to manage multiple subscriptions yourself. 4) Cross-Session Communication AgentSession instances are isolated — one agent cannot directly call methods on another agent’s session. Use RPC/data channels (or LiveKit data messages/topics) for cross-agent communication and to request actions like interrupts or reply generation. 5) Muting Participants Programmatically You can mute a remote participant’s published audio via server-side APIs that act with room admin privileges (e.g., using

room.local_participant.mute_published_track()

or the RoomService mute functionality). An agent can mute others only if it has the necessary room/admin permissions; otherwise perform the mute from a backend process that holds those privileges. 6) Silent Agent Pattern Recommended: disable TTS for the observer by setting

tts=None

(or equivalent) on the AgentSession so it does not publish audio. You don’t need a dummy TTS; disabling TTS avoids creating/publishing an audio track. 7) Agent Subscribing to Another Agent's Audio Yes. An agent can subscribe to another agent’s published audio track using standard LiveKit subscription mechanics. With RoomIO you can target the other agent via

set_participant()

(or use low-level participant events/subscriptions for more control). 8) Detecting Agent Turn Completion Watch

agent_state_changed

transitions (e.g., from "speaking" to "listening"/"thinking") to detect turn completion. You can also await the

SpeechHandle

returned by

say()

or

generate_reply()

to know when the speech finishes. 9) Update Chat Context from External Source You cannot directly manipulate another session’s chat context from outside. Have the observer signal the main agent via data messages/RPC; the primary agent then injects the system/user message into its own context and continues the flow. 10) Agent Dispatch vs Same-Process Creation • Creating a second AgentSession in the same entrypoint: runs both agents in one worker process — good for tight coupling, shared memory, simpler coordination. • Using `api.agent_dispatch.create_dispatch()`: spawns a separate agent worker — provides isolation, independent scaling, and fault isolation. Choose same-process for low-latency, tightly integrated agents; choose dispatch when you need independent scaling, resilience, or strict isolation. If you want, I can: • Show a short Python example for creating two AgentSessions in one process, or • Show how to mute a participant via the RoomService API, or • Outline a simple data-channel RPC pattern for agent-to-agent commands. Which would you like next? Sources: Virtual avatar plugins | LiveKit Docs | Job lifecycle | LiveKit docs | How to set a fixed participant for agent audio processing | Agents Framework | livekit/livekit_composite | DeepWiki | Bringing AI avatars to voice agents

ince muting requires admin privileges, ask: For muting a SIP participant: Can I use ctx.api.room.mute_published_track() from within an agent? What permissions does the agent need? Or should I call the LiveKit Server API (RoomService) from Python? And for the data channel pattern: Can you show a simple data-channel pattern for agent-to-agent commands? I want Observer Agent to send commands like {"action": "interrupt", "hint": "..."} to the Main Agent.

Regarding muting a SIP participant • Yes — muting a participant requires room/admin privileges. An agent can call

ctx.api.room.mute_published_track()

only if that agent’s session has the necessary room admin permissions (i.e., it is authorized to perform server-side mute actions). If the agent does not hold those privileges, perform the mute from a backend process that has room admin rights using the LiveKit Server (RoomService) API from Python. • In practice: if your agent process is granted admin-level scopes/credentials, calling

ctx.api.room.mute_published_track()

from within the agent is fine. If not, call the RoomService mute endpoint from a privileged backend. Data-channel pattern for agent-to-agent commands 1. Overview • Use LiveKit data messages/topics to send JSON commands between agents. • Observer publishes a data message to the room with a command payload; the Main Agent subscribes to data messages and handles commands. 1. Minimal pattern (conceptual Python pseudocode) Observer agent sends command:

cmd = {"action": "interrupt", "hint": "urgent safety issue"}
await ctx.room.publish_data(json.dumps(cmd).encode(), kind="reliable")

Main agent receives and handles:

# set up a data message handler when session/room connects
def on_data_message(message, participant):
    payload = json.loads(message.data.decode())
    if payload.get("action") == "interrupt":
        # trigger interrupt handling within this agent
        await agent_session.interrupt()
        # optionally send ack back
        ack = {"action": "ack", "for": payload}
        await ctx.room.publish_data(json.dumps(ack).encode(), kind="reliable")

ctx.room.on("data_received", on_data_message)

Notes and best practices • Use

kind="reliable"

for control messages that must arrive; use

kind="lossy"

for non-critical events. • Include sender identity and correlation IDs in the payload for authorization and tracing. • Validate incoming commands to avoid unauthorized actions. • For richer RPC behavior, implement request/response with unique IDs and timeouts (observer sends

{id, action}

, main agent replies with

{id, result}

). If you want, I can produce a more complete code example that wires handlers into AgentSession lifecycle and shows request/response with timeouts. Would you like that?