hey, how I am supposed to handle the tts connections for multiple voice agents in a single system. When and how to close the connections cleanly without getting

WARNING livekit.agents - failed to synthesize speech, retrying in 0.1s

Thanks for reaching out, Clovis. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

what is the best practices when setting up tts for a multi agent system? should they all share the same tts connection like they do for the chat_ctx, should it be a new one each time. how do deal with it during handoffs?

1. Shared vs. New TTS Connections • Shared Session-Level TTS: Hey there! When you want all your agents to sound like they're part of the same team, configure TTS at the session level. It's like giving everyone the same friendly uniform – super helpful for keeping things smooth when agents pass the conversation baton. • Agent-Specific TTS: But wait, sometimes you want each agent to have their own unique voice. No problem! You can totally override the session TTS to give individual agents their own special sound. Think of it like giving each team member a distinct personality. 2. Connection Management • Pro tip: Use the

FallbackAdapter

– it's like having a backup buddy who's always got your back. This little gem manages multiple TTS instances and helps you recover smoothly if something goes wrong. • For those mission-critical systems, set up those fallback mechanisms and retry limits. It's like having a safety net for your conversations! 3. Handoff Considerations • Maintain Consistency: When you're passing a conversation between agents, try to keep the voice feeling familiar. It's like continuing a chat with the same friend, just a different person from the team. • Differentiate Agents: Want to help users know who they're talking to? Give each agent a slightly different voice – it's like giving them their own communication style. • For those realtime API wizards, just set

tts=None

to dodge any potential voice conflicts. 4. Key Recommendations • Start simple! Go with session-level TTS – it's the easiest way to keep things smooth and consistent. • Only jump into agent-specific voices when you really need that special touch. • Think about your handoffs – will changing voices make the conversation feel more natural or just confuse users? • Be smart about those realtime APIs – they're convenient, but they might limit how creative you can get with voices. At the end of the day, choosing between shared or individual TTS is all about what works best for your specific setup. Trust your instincts and don't be afraid to experiment! Sources: livekit-agents/livekit/agents/voice/agent.py | main.py

I am not talking about the configuration here but the websocket connection itself, should they share the same?

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!