Instead of

AUTOSUBSCRIBING

the audio track of remote participant from the local participant, can the agent do it manually? When remote participant publishes their track does it get automatically and instantaneously subscribed without waiting for actual media start from remote to local participant?

Thanks for reaching out, aryan. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

explain with ref to python sdk

Hi there! To help you manually subscribe to audio tracks in LiveKit using the Python SDK, here’s a simple step-by-step guide you can follow: 1. Disable Auto-Subscribe: When connecting to the room, set

auto_subscribe=False

. This way, you get to decide exactly which tracks you want to subscribe to. 2. Subscribe to Tracks Manually: Use the

RemoteTrackPublication.set_subscribed(True)

method to subscribe to the specific tracks you’re interested in. You can do this when a participant joins or when a new track is published. 3. Process Subscribed Audio Tracks: Once you’ve subscribed, handle the

track_subscribed

event to work with the audio stream. Here’s an example to illustrate this:

async def main():
    room = rtc.Room()
    # Disable auto-subscribe
    await room.connect('LIVEKIT_URL', 'TOKEN', options=rtc.RoomOptions(auto_subscribe=False))

    # Subscribe to audio tracks when a participant joins
    @room.on("participant_connected")
    def on_participant_connected(participant: rtc.RemoteParticipant):
        for publication in participant.track_publications.values():
            if publication.track and publication.track.kind == rtc.TrackKind.KIND_AUDIO:
                asyncio.create_task(participant.subscribe(publication.track))

    # Handle newly published tracks
    @room.on("track_published")
    def on_track_published(publication: rtc.RemoteTrackPublication, participant: rtc.RemoteParticipant):
        if publication.track and publication.kind == rtc.TrackKind.KIND_AUDIO:
            asyncio.create_task(participant.subscribe(publication.track))

# Process subscribed audio tracks
@room.on("track_subscribed")
def on_track_subscribed(track: rtc.Track, publication: rtc.RemoteTrackPublication, participant: rtc.RemoteParticipant):
    if track.kind == rtc.TrackKind.KIND_AUDIO:
        audio_stream = rtc.AudioStream(track)
        # Process audio frames
        async for event in audio_stream:
            # Do something with event.frame
            pass
        await audio_stream.aclose()

This way, you have full control over which audio tracks your app receives and processes. If anything is unclear or you want to dive deeper, just let me know—I’m here to help! Sources: Receiving and publishing tracks | LiveKit Docs | Processing raw media tracks | LiveKit Docs | Subscribing to tracks | LiveKit Docs

Until Remote Participant subscribes to Local Participant's track (SIP Connection), does the remote participant hear ringing sound?

I totally understand how important it is for remote participants in a LiveKit SIP connection to hear that ringing sound. Just to share, they don't automatically hear it until they subscribe to a local participant's track. The audio playback during call setup is something you can configure, but it’s not on by default. If you'd like to have a ringing sound, you can enable the

play_dialtone

option when creating the SIP participant. This will play a dial tone (like the ETSI ringing) to everyone already connected in the room while the SIP call is connecting. Just keep in mind, this tone won’t reach the remote SIP participant until a two-way media connection is set up. Please feel free to reach out if you'd like me to walk you through setting this up or if you have any other questions—I’m here to help! Sources: SIP Integration | livekit/livekit_composite | DeepWiki | SIP participant | LiveKit Docs | Make outbound calls | LiveKit Docs | pkg/sip/inbound.go | livekit/livekit_sip.pb.go

Explain the event:

@_room_.on("room_updated")

, what info does it store? when is it called in a call lifecycle?

I'd love to help you out! Could you please share which SDK or programming language you're working with (like Python, JavaScript, etc.)? That way, I can give you the most accurate and helpful details about the

@room.on("room_updated")

event and how it fits into the call lifecycle.

python

I'm glad you're exploring the LiveKit Python SDK! The

@room.on("room_updated")

event is a helpful feature that gets triggered whenever there's an update to the room's state during your call. This can include changes to things like: • Room metadata: Any custom metadata you've associated with the room. • Number of participants: When participants join or leave, this updates accordingly. • Recording status: Whether the room is currently being recorded. • Room settings: Adjustments to properties such as

empty_timeout

,

max_participants

, and more. Whenever this event fires, the internal

RoomInfo

object refreshes with the latest room data. You can easily access properties like

room.name

,

room.metadata

,

room.num_participants

, and

room.is_recording

to get the most up-to-date information. Here's a simple example to illustrate how you might use it:

@room.on("room_updated")
def on_room_updated():
    print(f"Room updated - Name: {room.name}, Participants: {room.num_participants}, Recording: {room.is_recording}")

This event is really handy for responding to changes in real time—like updating your UI or triggering specific actions based on what's happening in the room. If you have any questions or want me to walk you through it further, just let me know! I'm here to help. Sources: livekit-rtc/livekit/rtc/room.py | livekit-ffi/src/livekit.proto.rs

Consider the case where a dialer first calls livekit's voice agent and then human. My hunch is that the dialer is essentially transmitting audio to the customer. So in the room, we have voice agent and the dialer, will room_updated event fire when the customer has picked up the call?

Absolutely! The

@room.on("room_updated")

event will indeed fire when the customer picks up the call in the scenario you described. This happens because the event is designed to trigger whenever there's a change in the room's state, like when the number of participants (

num_participants

) goes up. Here’s a quick overview of your workflow: 1. The dialer calls the voice agent (connected via SIP). 2. When the customer answers, they join the room, which increases the participant count. This increase in

num_participants

or any other updates to room-level properties (such as metadata) will activate the

room_updated

event. You can rely on this event to keep an eye on these changes in real time. For instance:

@room.on("room_updated")
def on_room_updated():
    print(f"Room updated. Participants: {room.num_participants}")

If you want to track exactly who joined, you might find the

participant_connected

event helpful as well:

@room.on("participant_connected")
def on_participant(participant):
    print(f"Participant joined: {participant.identity}")

Feel free to reach out if you’d like me to walk you through anything else or if you have more questions—I'm here to help! Sources: livekit-agents/livekit/agents/voice/room_io/room_io.py | packages/livekit-rtc/src/proto/room_pb.ts | pkg/rtc/room.go | livekit/src/room/mod.rs | livekit-rtc/livekit/rtc/room.py

Buty I have 3 participants max in the room during a call: agent, dialer and egress. So when the customer picks up, I can see no new event from the liveikit dashboard -> session --> events. The events display: Particpant joined, Participant active, Participant left timestamps and event payload for all the 3 participants. But customer picks up the call behind the dialer outside of room, will we get any event in that case?

I understand that when a customer picks up a call outside the room (like behind a dialer), LiveKit doesn’t trigger a new event in the room itself, since the customer isn’t directly joining the room. But don’t worry! You can still keep an eye on the call state using the

sip.callStatus

attribute of the SIP participant. This attribute shows you exactly what’s happening with the call: • dialing: The call is being initiated. • ringing: The call is ringing on the recipient’s device. • active: The customer has picked up the call. • hangup: The call has ended. For instance, you could poll the

sip.callStatus

attribute to spot when the call becomes active, like this:

while perf_counter() - start_time < 30:
    call_status = participant.attributes.get("sip.callStatus")
    if call_status == "active":
        print("Customer has picked up the call")
        break
    elif call_status == "hangup":
        print("Call ended")
        break
    await asyncio.sleep(0.1)

Also, LiveKit sends webhook events such as

participant_joined

or analytics events like

SIP_CALL_STARTED

when the call connects. These are great tools to help you track the call lifecycle even outside the room. If you’d like, I’m here to help you set up these mechanisms or guide you through implementing them in your application—just let me know! Sources: src/room/events.ts | Making calls using SIP | LiveKit Docs | Making calls using SIP | LiveKit Docs | SIP Integration | livekit/livekit_composite | DeepWiki | pkg/telemetry/events.go

How does one pre-load the agent in livekit room? Lets consider the other scenario where dialer calls the human and then the livekit voice agent. Can I already keep the room connected and agent loaded on it? If so, how? Currently, there is a sequential flow in ctx.connect ---> wait_for_participant ---> initialisation of agent (VoicePipeline/Multimodal) -->agent start session

To help you pre-load a LiveKit voice agent in a room before a call starts, here are some friendly approaches you can try: 1. Pre-initialize Agent Resources: You can use the

prewarm

function to preload models like VAD right when the worker process starts. This really helps cut down on any delays when the agent gets dispatched. 2. Dispatch Agent Early: Another great way is to dispatch the agent early during room or token creation. The agent can connect to the room and patiently wait for participants by using

ctx.wait_for_participant()

. 3. Keep Agent Connected: You might also keep the agent connected and active in the room. For example:

await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
await ctx.wait_for_participant()
...
agent.start(ctx.room)

In your case, where the dialer calls the human first and then the agent, here’s a simple way to handle it: 1. Create the room and dispatch the agent at the same time. 2. Have the agent connect and wait patiently in the room. 3. Once the human joins, the agent will detect them and start the session. This way, your agent will be nicely pre-loaded and ready to go with minimal waiting time. If you’d like, I’m here to help with any further questions or specific details you might need! Sources: Web and mobile frontends | LiveKit Docs | External data and RAG | LiveKit Docs | agent/main.py | examples/src/realtime_turn_detector.ts