explain with example code how best to configure elevenlabs tts with livekit

Thanks for reaching out, Sunil. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I would like detailed configuration for a realistic human like audio in the agent session. give me python example code

I'd love to help you get the perfect "realistic human-like audio" setup! Could you share a bit more about what you're hoping to adjust? For instance: • Are you interested in particular voice models or tweaking things like pitch, tone, or speed? • Would you like to use SSML for custom pronunciation or maybe switch languages dynamically? • Do you have a specific use case or scenario in mind for the AgentSession? Feel free to share whatever details you have, and I'll make sure to guide you in the best way possible.

I am particularly interested in elevenlabs. make recomemndation for the most human like voice and what configuration is best to be used in agent session as text to speech node

I'd love to help you find the most human-like voice that fits your needs perfectly! Could you share a bit more about what you're looking for? For instance: • Is there a particular language or accent you have in mind? • Do you have any preferences regarding voice characteristics, like gender, tone, or style? • Are there specific situations or uses (such as customer support or narration) where this voice will be used? Knowing these details will really help me suggest the best ElevenLabs voice and settings for your AgentSession. Looking forward to hearing from you!

yes...the requirement is for a english language assistant that is professional and empathitic. The agent is a debt collection agent that makes outgoing calls via sip

Hi there! Let’s walk through how to set up ElevenLabs to create a professional and empathetic English voice for a debt collection agent in an AgentSession with SIP. I’m here to guide you every step of the way: 1. Installation and Authentication First, let’s get the plugin installed and authenticate your access:

pip install "livekit-agents[elevenlabs]~=1.2"

Then, add your ElevenLabs API key to a

.env

file like this:

ELEVEN_API_KEY=<your-api-key>

2. Voice and Model Selection Next, you’ll want to configure a voice that sounds warm and empathetic. Here’s an example using the

TTS

class:

from livekit.plugins import elevenlabs

tts_instance = elevenlabs.TTS(
    voice_id="EXAVITQu4vr4xnSDxMaL",  # "Bella", known for warm tone
    model="eleven_multilingual_v2",
    voice_settings=elevenlabs.VoiceSettings(
        stability=0.65,
        similarity_boost=0.6,
        style=0.4,
        use_speaker_boost=True,
        speed=0.95  # Slightly slower for empathy
    ),
    language="en"
)

The

voice_id

"EXAVITQu4vr4xnSDxMaL" corresponds to "Bella," a voice that’s recognized for its warm and clear qualities—perfect for your use case. 3. Programmatically List and Select Voices If you want to explore and pick the best voice, you can use

list_voices()

like this:

async def select_empathetic_voice():
    tts = elevenlabs.TTS()
    voices = await tts.list_voices()
    candidate_voices = [v for v in voices if "bella" in v.name.lower() or "jessica" in v.name.lower()]
    return candidate_voices

4. Integrate with AgentSession and SIP Once you have your voice ready, you can integrate it into a telephony-capable

AgentSession

like so:

session = AgentSession(
    tts=tts_instance,
    stt=deepgram.STT(),  # Example STT
    llm=openai.LLM(),
    vad=silero.VAD.load(),
    allow_interruptions=False
)

5. Enhance Empathy Through LLM Instructions To make sure your agent responds with genuine empathy, you can provide clear instructions to the language model:

instructions = """
You are a compassionate and professional debt collection agent.
Acknowledge the user’s financial stress, offer flexible payment options,
and always maintain a respectful and understanding tone.
Avoid aggressive language or pressure.
"""

6. Test and Evaluate Voice Output • A/B Test Voices: Try different `voice_id`s in test calls to see which one feels most empathetic. • Use SSML for Pronunciation: Enable

enable_ssml_parsing=True

to ensure financial terms are pronounced correctly. • Enable Transcription Alignment: Set

use_tts_aligned_transcript=True

in

AgentSession

for synchronized transcription. If you have any questions or need help with any part of this setup, please don’t hesitate to reach out. I’m here to support you! Sources: ElevenLabs TTS integration guide | LiveKit Docs | ElevenLabs integration guide | LiveKit Docs | examples/warm-transfer/warm_transfer.py | plugins/elevenlabs/src/tts.ts