Can i have multiple agent sessions in one script?

Thanks for reaching out, Derek. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Can each of these sessions have their own virtual avatar?

Absolutely! Each agent session in LiveKit can have its very own virtual avatar, which brings a lifelike video presence to your voice AI agents. You can easily integrate avatars using supported providers like Anam, Beyond Presence, bitHuman, Hedra, Simli, and Tavus. Here’s a simple way to get this set up: 1. First, install the plugin from your chosen avatar provider and set up your API keys. 2. Next, create an

AgentSession

with the speech-to-text (STT), language model (LLM), and text-to-speech (TTS) settings that fit your needs. 3. Then, create an

AvatarSession

(like

hedra.AvatarSession

,

tavus.AvatarSession

, or

bithuman.AvatarSession

) and link it to your

AgentSession

. 4. Finally, start the

AvatarSession

first, and then launch the

AgentSession

with audio output turned off, since the avatar takes care of the audio. The avatar’s video feed will automatically appear on the frontend through LiveKit’s client SDKs. Plus, our starter apps for web and mobile already support avatar rendering out of the box. If you’d like, I’m here to help with more details or examples—just let me know! Sources: Web and mobile frontends | LiveKit Docs | Virtual avatar integrations | LiveKit Docs | Web and mobile frontends | LiveKit Docs | avatars/hedra/dynamically_created_avatar/agent.py | livekit-plugins/livekit-plugins-simli/README.md

How do I switch between multiple agent sessions that each have their own virtual avatar?

To help you switch between multiple agent sessions, each with their own virtual avatar, here are a couple of friendly approaches you can try: 1. Switching Agents: You can use the

update_agent()

method to smoothly switch the active agent within the same session. Alternatively, returning a new agent instance from a function tool will trigger an automatic handoff, making the process seamless. 2. Managing Avatars: Each agent session can have its own virtual avatar! Just create and start an

AvatarSession

before the

AgentSession

. Each avatar is linked to its session, and the frontend will automatically show the avatar’s video feed using LiveKit’s client SDKs. It’s good to know that avatars are generally tied to the session level, so there isn’t built-in support for switching avatars dynamically mid-session. If having different avatars for each agent is important, you might want to set up workflows where each agent session comes pre-configured with its own avatar. If you’d like, I’m here to provide more details or help you implement this setup—just let me know! Sources: Tavus virtual avatar integration guide | LiveKit Docs | Workflows | LiveKit Docs | README.md | Virtual avatar integrations | LiveKit Docs | Anam virtual avatar integration guide | LiveKit Docs

What does the update_agent() method do?

The

update_agent()

method in LiveKit is quite handy and serves two main purposes: 1. In the Python SDK: It lets you smoothly switch the active agent during a session. For instance, you can easily replace the current agent with a new one, like moving from

ShortAgent()

to

LongAgent()

. This makes it super convenient to handle multi-agent workflows without any hiccups. 2. In the Cloud Agent API and CLI: It helps you update an agent's metadata and secrets—things like regions or credentials—and restarts the agent workers without disrupting any active sessions. This is especially useful for making infrastructure-level updates to your deployed agents. If you'd like, I’m here to walk you through more details on either of these uses. Just let me know! Sources: multi-agent/long_or_short_agent.py | cmd/lk/agent.go | livekit/livekit_cloud_agent.twirp.go | Agent deployment CLI reference | LiveKit Docs | Workflows | LiveKit Docs

Can i update the tts to model when i update the agent as well

Thanks for your question! The

update_agent()

method doesn’t directly support updating the TTS model, but don’t worry — you can still update the TTS model dynamically during runtime using the

update_options()

method available in some TTS plugins. For instance, plugins like Cartesia, Groq, LMNT, and ElevenLabs let you switch models easily by calling

update_options(model="desired_model")

. If your TTS plugin doesn’t support dynamic model updates, a good approach is to reinitialize the TTS plugin with the model you want and then pass it to the agent during setup. If you’d like, I’m here to help with specific examples or any other questions you have! Sources: AI Voice Assistant Quickstart | LiveKit Docs | livekit-agents/livekit/agents/voice/agent_activity.py

Give me an example

Here’s a friendly example to help you update the TTS model dynamically using the

update_options()

method. This one uses the Cartesia TTS plugin:

from livekit.plugins import cartesia

# Initialize TTS with a default model
tts_instance = cartesia.TTS(model="sonic-1")

# Update the TTS model and other parameters during runtime
tts_instance.update_options(
    model="sonic-2-2025-03-07",
    speed=2.0,
    emotion=["joy", "confidence"]
)

Here, the TTS model switches to "sonic-2-2025-03-07" and updates the speed and emotion settings. These changes will take effect on the very next utterance. If you’d like, I’m happy to provide examples for other TTS plugins or help you with anything else—just let me know! Sources: livekit-plugins/livekit-plugins-azure/livekit/plugins/azure/tts.py | Hume TTS integration guide | LiveKit Docs

Does the hedra virtual avatar have a update_options method?

Thanks for reaching out! I wanted to share that the Hedra virtual avatar integration doesn’t have an

update_options

method in its

AvatarSession

class. The parameters like

avatar_id

,

avatar_image

, and

api_key

are set when you start the session and can’t be changed on the fly. If you need to update any of these, the best way is to create a new

AvatarSession

instance with the settings you want. Please feel free to ask if you’d like me to walk you through this or if there’s anything else I can help with! Sources: livekit-plugins/livekit-plugins-gladia/livekit/plugins/gladia/stt.py | Hedra virtual avatar integration guide | LiveKit Docs

How does stop response work

I'm happy to help clarify the "stop response" functionality in LiveKit, which can actually mean two different things depending on what you're working on: 1. Stopping Egress Operations: This is how you can end an ongoing recording or streaming session. For example, by sending a

StopEgressRequest

with the

egress_id

of the session, you can stop it smoothly. This comes in handy when you want to manage recordings or streams programmatically. 2. Stopping Agent Responses: Within LiveKit Agents, the

StopResponse

exception helps prevent an agent from replying during a conversation. This is especially useful in cases like push-to-talk interfaces, where you want to avoid responses to non-speech input. If you'd like, I can share some specific examples or dive deeper into either of these—just let me know! I'm here to make this as clear and easy as possible for you. Sources: Docs::Agents | Pipeline nodes and hooks | LiveKit Docs | agents/src/voice/agent.ts | livekit-api/livekit/api/egress_service.py | livekit/livekit_egress.twirp.go

I'm talking about stopResponse for agents and how do I use it

Hi there! I’d love to help you understand how the

StopResponse

exception works for agents in LiveKit. It’s a handy tool that lets you prevent an agent from generating a reply during a conversation. This can be really useful in situations like push-to-talk interfaces or when you want to avoid responses to nonspeech or irrelevant input. Here’s a simple way to use it: • In Python:

from livekit.agents import StopResponse

# Example: Suppress response if the message has no text content
if not new_message.text_content:
    raise StopResponse()

• In Node.js:

const { StopResponse } = require("@livekit/agents");

// Example: Suppress response if the message has no text content
if (!newMessage.textContent) {
  throw new StopResponse();
}

When you raise

StopResponse

, the agent immediately stops generating a reply, so no unnecessary messages are sent. If you have any questions or want me to walk you through more examples, just let me know—I’m here to help!