I created a new LiveKit cloud account today, built an agent using the visual editor. However when I join a room the agent is not joining and I am not seeing why? Am I missing something basic?

Thanks for reaching out, Andres. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Ok I figured out, its only joining in the playground if I set the Agent Name. How can I get it to join any call by default?

Thanks — I’m glad you found the Agent Name clue. You’re asking how to have the agent join any call by default (so you don’t need to set the Agent Name each time). Here’s how to do that and what to check. Short answer • Use automatic dispatch for the agent and ensure it’s Deployed. Automatic dispatch causes the agent to join new rooms by default without specifying an agent name. Steps to enable automatic joining 1. Open the Agent Builder for your agent in the LiveKit console. 2. Confirm the agent’s dispatch mode is set to Automatic dispatch (this is the visual editor’s default). 3. Click Deploy agent (top-right) so the deployment is active — agents in preview or not deployed will not join production rooms. 4. When testing, create/join a new room (automatic dispatch targets newly created rooms). The agent will auto-join that new room. 5. Use the Test Agent button to open the pre-configured frontend — this verifies automatic join behaviour quickly. Notes and common pitfalls • Automatic dispatch does not retroactively join existing rooms. If you join an already-created room, the agent may not be dispatched. Create a new room to test. • The agent name only needs to be set if you’re using explicit dispatch (targeting a specific agent). If you prefer per-room targeting, you can dispatch by agent name instead of automatic mode. • If the agent doesn’t join after deployment, check the Agents dashboard for the agent’s status and inspect Agent insights / Sessions logs for dispatch or connection errors. If you’d like, I can walk you through: • Verifying the agent’s dispatch mode and deployment in your project (tell me the agent status you see), or • Changing from automatic to explicit dispatch and how to trigger an agent for an existing room. Which would you like to do next?

with the LK JS sdk can Is et the AgentName?

Thanks — I’m responding to your question about whether you can set the AgentName from the LiveKit JS SDK. Short answer • No — client SDKs (browser/mobile

livekit-client

or

@livekit/rtc-node

) cannot set

agentName

. Agent dispatch is controlled by the agent process/server, not by participants connecting from client code. What you can do • Set

agentName

in the agent server (Node) using

ServerOptions

from

@livekit/agents

. This makes the agent an explicitly-named agent that will only join when dispatched to that name. Example (Node.js agent):

import { ServerOptions } from '@livekit/agents';

const opts = new ServerOptions({
  agentName: 'my-test-agent',
  // other options...
});

// run your agent with these options
cli.runApp(opts);

If you want the agent to join calls by default (without naming/explicit dispatch) • Use automatic dispatch in the agent builder or agent server configuration and ensure the agent is Deployed. Automatic dispatch causes the agent to join new rooms without needing an

agentName

from the client. If you want help with either option, tell me: • Do you prefer automatic dispatch (agent joins new rooms by default) or explicit dispatch (agent only joins when named)? • If explicit, do you need the code to dispatch an agent to an existing room or guidance on configuring the agent server? I can walk you through the next steps for your preferred approach. Sources: Agent dispatch | LiveKit Documentation | Server options | LiveKit Documentation | Agent session | LiveKit Documentation

I am not seeing how to set the dispatch rule from the UI. I see that it set the angent_name and wont let me delete it. @breezy-insurance-66772.rtc_session(agent_name="Andr3s")

Thanks — I’m responding to your note that the agent has an

agent_name

set (

@FlashCall Server.rtc_session(agent_name="Andr3s")

) and you can’t remove it from the UI. Quick clarification so I give the right steps: • Are you editing this agent from the LiveKit Cloud console (Agent Builder) visual editor, or from a local agent server config (your Node agent code)? • Do you want the agent to use automatic dispatch (join new rooms by default) or explicit dispatch (only join when named)? What’s happening • The

agent_name

you’re seeing is an explicit agent identifier. When an agent has an

agent_name

configured, it is treated as an explicitly named agent and won’t be auto-dispatched unless a dispatch rule or call explicitly targets that name. That setting can come from the agent’s server configuration or from a rule that binds the agent to sessions. • The console may prevent deleting or changing the name if the agent was deployed with that name in the agent server options or in code (for example using

ServerOptions.agentName

), because the running agent process is the canonical source of that value. How to remove or change it • If the name was set in the visual Agent Builder and is editable there: ◦ Open the Agent Builder for that agent, switch to the deployment/config section, and remove or clear the Agent Name field, then Deploy agent again. • If the name was set in your agent server code (Node/`@livekit/agents`): ◦ Update the

ServerOptions

(or wherever

agentName

is defined) to remove or change the value, then restart/redeploy the agent so the new config takes effect. Example:

const opts = new ServerOptions({
      // remove agentName or set to undefined to allow automatic dispatch
      // agentName: undefined,
    });

• After restarting the agent without

agentName

, the agent will no longer be an explicitly-named agent. • If the name is enforced by a dispatch rule or routing config: ◦ Edit or remove the dispatch rule that targets

Andr3s

so the agent is not required by name. Dispatch rules may live in your project settings or in any external system that calls the AgentDispatchService. If you want automatic dispatch (agent joins any new room by default) • Ensure the agent has no

agentName

configured and is set to Automatic dispatch in the builder, then Deploy agent. Automatic dispatch only targets newly created rooms — it won’t retroactively join existing rooms. Next steps I can take for you • Walk you step-by-step in the console: tell me whether you see the Agent Builder or the agent server was deployed from code. • If it’s from code, paste the small snippet where

ServerOptions

or

agentName

is set and I’ll point to the exact change. • If it’s in the console but the field is locked, tell me the agent’s dashboard status and I’ll guide you where to redeploy or how to clear the name. Which environment is this from — the Cloud console visual editor or your agent server code?

How can I make sure Cartesia knows how to pronounce certain words I don't want it to spell out?

Thanks — I’m responding to your question about preventing Cartesia from spelling out words and ensuring it pronounces them the way you want. Short answer • Use SSML (Speech Synthesis Markup Language) with phoneme, say-as, or a lexicon to control pronunciations. Cartesia supports SSML tags so you can force phonetic pronunciations or specify how acronyms/numbers should be read. How to do this (practical options) • Use the <phoneme> tag to give a phonetic pronunciation (IPA or other supported alphabets). ◦ Example:

<speak>
      Please call our <phoneme alphabet="ipa" ph="ˈeɪpiˌaɪ">API</phoneme> support.
    </speak>

• This prevents the TTS from spelling “API” and pronounces it as “ay-pee-eye.” • Use <say-as> for common formats so they’re read naturally (dates, times, numbers, acronyms). ◦ Example:

<speak>
      Your flight is on <say-as interpret-as="date">2026-03-10</say-as>.
    </speak>

• Use a lexicon for broader, reusable pronunciation rules. ◦ Lexicons let you define word → pronunciation mappings you can reuse across TTS requests rather than embedding phonemes in every string. Where to apply the SSML • Wrap the spoken text you pass to Cartesia’s TTS with the SSML markup before sending it to the TTS plugin in your agent. • Ensure the TTS request indicates SSML input (some integrations require a flag or content type to parse SSML). Extra tips • Test IPA phonetics in small snippets to confirm the pronunciation — different TTS voices can have slight variations. • For many acronyms, try

<say-as interpret-as="characters">

or phoneme alternatives depending on desired outcome. • If you have a list of words you want consistently pronounced, a lexicon is cleaner than repeating phonemes. If you want, I can: • Help convert a few example words you’re seeing spelled out into SSML phonemes or say-as tags. Paste the words or sample sentence and I’ll provide the SSML. • Show how to include SSML in your agent’s TTS call (if you tell me whether you’re sending TTS from the visual agent builder or from code). Sources: Cartesia TTS plugin guide | LiveKit Documentation | Agent speech and audio | LiveKit Documentation

How do I modify my python agent to include those?

Thanks — I’m responding to your request for concrete steps to add SSML (phonemes, say-as, lexicons) into a Python LiveKit agent so Cartesia (or another TTS plugin) pronounces words correctly. Below are concise, actionable steps and code snippets you can drop into your Python agent. I’ll cover: • confirming SSML support, • enabling SSML parsing for common providers, • sending SSML via session.say(), • a simple preprocessing helper to inject phonemes automatically, • and an optional tts_node override for automatic SSML injection. 1) Confirm your TTS provider supports SSML • Cartesia, ElevenLabs, Amazon Polly, Google Cloud TTS, and Azure Speech support SSML. Make sure your chosen plugin is configured to accept SSML input. 2) Enable SSML parsing (if required by provider) • ElevenLabs example:

from livekit.plugins import elevenlabs

tts = elevenlabs.TTS(
    voice_id="EXAVITQu4vr4xnSDxMaL",
    enable_ssml_parsing=True  # required for ElevenLabs
)

• Amazon Polly example:

from livekit.plugins import aws

tts = aws.TTS(
    voice="Ruth",
    text_type="ssml"  # tells Polly to treat text as SSML
)

(Other providers usually parse SSML automatically when SSML tags are present.) 3) Send SSML directly with session.say() • Wrap the text in a <speak> tag and include <phoneme> / <say-as> where needed:

await session.say(
    text='<speak>Please call our <phoneme alphabet="ipa" ph="eɪ pi aɪ">API</phoneme> team.</speak>',
    allow_interruptions=True
)

This prevents “API” from being spelled out. 4) Helper to automatically replace words with SSML phonemes • Useful if you have many words to normalize consistently:

import re

pronunciation_map = {
    "LiveKit": '<phoneme alphabet="ipa" ph="laɪv kɪt">LiveKit</phoneme>',
    "API": '<phoneme alphabet="ipa" ph="eɪ pi aɪ">API</phoneme>',
}

def apply_ssml_pronunciations(text: str) -> str:
    for word, replacement in pronunciation_map.items():
        text = re.sub(rf'\b{re.escape(word)}\b', replacement, text, flags=re.IGNORECASE)
    return f'<speak>{text}</speak>'

# Usage:
raw = "Welcome to LiveKit. Please contact API support."
ssml = apply_ssml_pronunciations(raw)
await session.say(text=ssml)

5) Optional: override tts_node to inject SSML automatically (advanced) • If you want every outgoing TTS chunk processed automatically, override

tts_node

to transform text before it reaches the TTS plugin:

class MyAgent(Agent):
    async def tts_node(self, text_stream, model_settings):
        async def apply_ssml(input_stream):
            async for chunk in input_stream:
                chunk = chunk.replace("API", '<phoneme alphabet="ipa" ph="eɪ pi aɪ">API</phoneme>')
                yield f'<speak>{chunk}</speak>'

        async for frame in await super().tts_node(apply_ssml(text_stream), model_settings):
            yield frame

• Note: ensure your provider is OK with receiving SSML-wrapped chunks and that you don’t accidentally double-wrap already-SSML text. 6) Testing and tips • Test short phrases first to confirm pronunciation. • For repeated/large lists, prefer lexicons (if provider supports) to keep your text clean. • If a word still sounds wrong, tweak its IPA in the phoneme tag—voices interpret IPA slightly differently. • Use <say-as> for dates, numbers, or acronyms if that gives better results for your use case. If you want, tell me: • Which TTS plugin you’re using (Cartesia, ElevenLabs, Polly, etc.), and • A couple of specific words or lines that are being spelled out. I’ll convert those exact examples into ready-to-use SSML for your agent and show the minimal code change needed. Sources: ElevenLabs TTS plugin guide | LiveKit Documentation | Amazon Polly TTS plugin guide | LiveKit Documentation | Google Cloud TTS integration guide | LiveKit Documentation | Azure Speech TTS plugin guide | LiveKit Documentation | Speechify TTS plugin guide | LiveKit Documentation

Example "STRATIS" should be "strat-iss" I am using: tts=inference.TTS( model="cartesia/sonic-3", voice="86e30c1d-714b-4074-a1f2-1cb6b552fb49" ),

Great — I’ll show you a small, practical change to your Python agent so Cartesia pronounces "STRATIS" as "strat-iss". Summary • Insert an SSML phoneme for STRATIS before it’s sent to the TTS engine. • With LiveKit Inference + Cartesia, include SSML in the text you pass to the TTS node; Cartesia will honor the phoneme. • Two options: (A) preprocess text before calling session.say(), or (B) override

tts_node

to automatically inject SSML for every TTS output. I’ll give both; pick the one you prefer. A) Minimal approach — preprocess before speaking • Use a small helper that converts occurrences of STRATIS into an SSML phoneme and then call session.say() with the SSML-wrapped text. Example:

import re

pronunciation_map = {
    "STRATIS": '<phoneme alphabet="arpabet" ph="S T R AE T IH S">STRATIS</phoneme>',
}

def apply_ssml_pronunciations(text: str) -> str:
    for word, replacement in pronunciation_map.items():
        text = re.sub(rf'\b{re.escape(word)}\b', replacement, text, flags=re.IGNORECASE)
    return f'<speak>{text}</speak>'

# Usage in your agent/session code:
raw_text = "Welcome to STRATIS. This is a demo."
ssml_text = apply_ssml_pronunciations(raw_text)
await session.say(text=ssml_text, allow_interruptions=True)

Notes: • I used ARPABET for the phoneme:

S T R AE T IH S

approximates “strat-iss.” If you want a slightly different vowel/timing, we can tweak the ARPABET sequence. • This is the simplest change and works well if you control the strings that get spoken. B) Automatic approach — override tts_node (recommended if many outputs should be fixed) • Override the agent’s

tts_node

so every text chunk is processed and SSML is injected automatically. Example pattern:

from typing import AsyncIterable
from livekit.agents import Agent
from livekit import rtc

class MyAgent(Agent):
    async def tts_node(self, text_stream: AsyncIterable[str], model_settings) -> AsyncIterable[rtc.AudioFrame]:
        # Collect full text (or process chunk-by-chunk if you prefer streaming)
        full_text = ""
        async for chunk in text_stream:
            full_text += chunk

        # Inject SSML phoneme
        corrected = full_text.replace(
            "STRATIS",
            '<phoneme alphabet="arpabet" ph="S T R AE T IH S">STRATIS</phoneme>'
        )

        ssml = f'<speak>{corrected}</speak>'

        # Delegate to the default tts_node implementation with the SSML text
        async for frame in await Agent.default.tts_node(self, [ssml], model_settings):
            yield frame

Notes: • This example buffers the full text; if you need strict streaming, we can adapt to process chunks while preserving SSML correctness. • Ensure the Inference

TTS

configuration uses your Cartesia model: tts = inference.TTS(model="cartesia/sonic-3", voice="86e30c1d-714b-4074-a1f2-1cb6b552fb49") Testing and tweaks • Deploy and test a short phrase like "Welcome to STRATIS" and listen. If it’s off, I can tweak the ARPABET phonemes (or switch to IPA) to better match "strat-iss". • If you have other words needing fixes, add them to the pronunciation_map and the same logic will handle them. Which option do you prefer — preprocess per utterance (A) or automatic tts_node override (B)? If you paste a few more words (or how you want them pronounced), I’ll provide the exact phoneme entries and a ready-to-drop-in snippet. Sources: Cartesia TTS plugin guide | LiveKit Documentation | Agent speech and audio | LiveKit Documentation

This is wrong I need to use a pronunciation_dict_id do you know how I can pass that in?

Thanks — I’m responding to your note that you need to pass a

pronunciation_dict_id

to Cartesia and want to know how to do that from your Python agent. Short answer • LiveKit’s Cartesia TTS integration does not document a

pronunciation_dict_id

parameter. The supported way to control pronunciation via LiveKit is SSML (phoneme tags) or provider-specific options that are exposed by the plugin/inference wrapper. • If Cartesia itself supports a

pronunciation_dict_id

on their API, it is not currently exposed in the LiveKit Cartesia plugin or the documented Inference TTS options. To help you further, I need a couple quick details so I can give the exact next step: 1) Which TTS constructor are you using in your agent right now? • Example A (Inference): tts = inference.TTS(model="cartesia/sonic-3", voice="86e30c1d-714b-4074-a1f2-1cb6b552fb49") • Example B (plugin): tts = cartesia.TTS(...) Please paste the exact code snippet you’re using. 2) Have you tried passing

extra_kwargs

or any other custom fields to the TTS constructor? If so, paste that snippet and any error or response you received. What I can do next (pick one) • Option A — Confirm & test a pass-through: I’ll show how to try sending

pronunciation_dict_id

via

extra_kwargs

in

inference.TTS

so you can test if the field is accepted (this is speculative — may be ignored if not supported). • Option B — Provide the robust SSML solution: I’ll give a ready-to-drop-in helper or a

tts_node

override to reliably make "STRATIS" speak as “strat-iss” using

<phoneme>

(works today with Cartesia via LiveKit). • Option C — Validate against Cartesia API: If you have a Cartesia doc or API link that explicitly shows

pronunciation_dict_id

, share it and I’ll map exactly how (and whether) LiveKit can pass it through or what change is needed. If you want to try Option A immediately, here’s the test snippet you can paste into your agent (speculative — may or may not be accepted by the integration):

from livekit.agents import inference

tts = inference.TTS(
    model="cartesia/sonic-3",
    voice="86e30c1d-714b-4074-a1f2-1cb6b552fb49",
    extra_kwargs={
        "pronunciation_dict_id": "your-dict-id"
    }
)

If this returns an error or the field is ignored, fallback to SSML is the supported path. Tell me which option you want to try and paste your current TTS construction code so I can give the exact next steps. Sources: Cartesia TTS plugin guide | LiveKit Documentation | Cartesia TTS | LiveKit Documentation | MiniMax TTS plugin guide | LiveKit Documentation

Can I add thinking sounds? there is sometimes a lot of silence between converations.

Thanks — I’m responding to your question about adding “thinking sounds” to reduce silence during agent pauses. Short answer • Yes — you can add thinking sounds using the BackgroundAudioPlayer in the Python agent. It can play built-in or custom audio clips while the agent is thinking (e.g., during LLM inference or tool calls). Quick example (Python) 1. Create a BackgroundAudioPlayer configured with thinking sounds. 2. Start it when your agent session begins. 3. Stop/close it when the session ends.

from livekit.agents import BackgroundAudioPlayer, AudioConfig, BuiltinAudioClip

# Configure thinking sounds (you can use built-in clips or point to files)
background_audio = BackgroundAudioPlayer(
    thinking_sound=[
        AudioConfig(source=BuiltinAudioClip.KEYBOARD_TYPING, volume=0.6, loop=True),
        # You can add multiple clips; player will choose/play based on config
    ],
    # optional: ambient_sound=[ AudioConfig(...) ] for continuous background noise
)

# Start playback once the agent session/room is ready
await background_audio.start(room=ctx.room, agent_session=session)

# ... later, when done:
await background_audio.aclose()

Notes and tips • Built-in options:

BuiltinAudioClip.KEYBOARD_TYPING

,

KEYBOARD_TYPING2

,

OFFICE_AMBIENCE

, etc. • You can use custom audio files (mp3/wav) by setting

AudioConfig(source="path_or_url_to_file")

. • Use

volume

,

loop

, and

probability

(if available) to tune how often and how loud thinking sounds play. • The BackgroundAudioPlayer targets “thinking” periods automatically (e.g., while waiting for model/tool responses). If you need precise control, start/stop the player around specific code sections (before inference start / after inference end). • Node.js: only ambient sound is supported currently; the Python SDK supports thinking_sound specifically. If you’d like, tell me: • Do you want built-in typing/ambient sounds or to use a custom audio file? • Should the thinking sound play only during LLM inference/tool calls, or whenever there’s >X seconds of silence? I can give the exact snippet tailored to your preferred audio source and when you want it to play. Sources: Agent speech and audio | LiveKit Documentation

It seems like the agent takes a while to join the room for the first time after a while is that expected?

Thanks — I’m responding to your question: “It seems like the agent takes a while to join the room for the first time after a while — is that expected?” Short answer • Yes. That initial delay is expected and is usually caused by a cold start: the agent process or container is spun up after a period of inactivity, models and dependencies are loaded, and network connections are established. Why this happens • LiveKit may scale agent instances to zero when idle to save resources. The first request after idle time triggers: ◦ launching the agent process/container, ◦ loading models and libraries (VAD, STT, TTS, etc.), ◦ establishing connections to LiveKit and external AI services. • Those steps add latency on the first join; subsequent joins are faster while the instance remains warm. How to reduce or eliminate the delay • Upgrade to an always-on plan (e.g., the Ship plan or higher) to avoid cold starts and keep agents warm. • Prewarm models during startup so they don’t load during the first interaction: ◦ Load VAD, TTS/voice models, or other heavy resources in a prewarm/init function. ◦ Example pattern: load the Silero VAD or other models into process userdata during initialization so inference is ready immediately. • Pre-download models and assets in your Docker image or startup script so runtime doesn’t fetch them. • Deploy in the same region as the AI services (LLM, STT, TTS) to reduce network latency. • For strict low-latency requirements, use always-on/enterprise features (dedicated capacity, regional deployments). Practical starter steps 1. Add a prewarm step in your agent init that loads any heavy models (VAD, etc.). 2. If you control deployment, add model download steps to your Dockerfile so startup doesn’t pull weights. 3. If you want instant joins and predictable latency, consider moving to an always-on deployment tier. If you’d like, I can: • Show a short code snippet to prewarm a VAD or another model in your Python agent. • Suggest exact Docker steps to pre-download models. • Help check your current agent logs to confirm the cold-start timing and where most time is spent. Which of those would you like to try first? Sources: Deployment reliability on LiveKit Cloud | LiveKit | How to match the latency of the homepage agent | LiveKit | How can I reduce latency in voice agents using STT, TTS and LLM? | LiveKit | examples/drive-thru | livekit/livekit_composite | DeepWiki | How much can LiveKit scale? | LiveKit