elevenlabs, cartesia and rime . list all the parameters for them , list them out each independet to the provider.

Thanks for reaching out, faiq. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

whats the difference between plugin parameters and via livekit tts inference ? I am using it in livekit btw ?

Thanks — I’ll explain the difference and what matters when you’re using these providers inside LiveKit. Short answer • Plugin parameters are the direct provider SDK/API options used by the ElevenLabs/Cartesia/Rime plugins. They map closely to that provider’s native features (voice IDs, voice_settings, emotion, sample_rate, etc.). • LiveKit TTS inference parameters are the parameters you pass to LiveKit’s inference layer. LiveKit forwards a smaller, normalized set (like

model

,

voice

,

language

, and an `extra_kwargs`/`modelOptions` bag) to the provider. Use plugin parameters when calling the provider directly; use the LiveKit inference shape when you send TTS requests through LiveKit. Key differences (practical) • Parameter shape ◦ Plugin: provider-specific field names (e.g., ElevenLabs uses

voice_id

, `voice_settings.stability`; Cartesia supports

voice

as an embedding array). ◦ LiveKit inference: consistent fields across providers — typically

model

,

voice

,

language

, plus

extra_kwargs

(called

modelOptions

in Node.js) to pass provider-specific options. • Required vs. optional ◦ Providers expose their own required fields. LiveKit requires the generic fields (

model

,

voice

for TTS) and accepts provider-specific options via

extra_kwargs

. • Normalization and routing ◦ LiveKit normalizes requests so your application can use the same top-level shape for different providers. Provider-specific options must go inside the extra kwargs so LiveKit can forward them unchanged. • Defaults and versions ◦ Plugin usage may rely on provider SDK defaults (e.g., default voices or model versions). When you go through LiveKit, specify the

model

you want (e.g.,

elevenlabs/eleven_flash_v2_5

) so LiveKit uses the correct backend model; otherwise LiveKit may apply its own defaults. • Error handling & validation ◦ Direct plugin calls surface provider-specific errors/validation. LiveKit may validate the top-level fields and forward provider errors from the backend; provider-specific validation still applies if you put options in

extra_kwargs

. • Use cases ◦ Use plugin parameters when you have direct provider credentials and want tight control or advanced features not exposed by LiveKit’s normalized fields. ◦ Use LiveKit inference parameters when you want a unified interface in LiveKit (single place to switch providers, consistent call patterns, and in-call streaming integration). How to pass provider-specific options via LiveKit • Top-level (required):

model

(e.g.,

cartesia/sonic-3

),

voice

,

language

(optional). • Provider-specific options: put them into

extra_kwargs

(or

modelOptions

in Node.js). Example: ◦ extra_kwargs: { speed: 1.2, emotion: "happy", volume: 0.9 } What I recommend for your LiveKit setup • If you want consistent code across providers: call LiveKit inference with

model

,

voice

,

language

, and pass provider-only fields in

extra_kwargs

. • If you need a provider feature that LiveKit’s inference doesn’t forward, call the provider plugin directly with its native parameters (useful for advanced voice_settings or low-level controls). If you want, I can: • Show a concrete example request for each provider through LiveKit (with

extra_kwargs

) so you can copy/paste into your app. • Compare a direct plugin payload vs. the LiveKit inference payload side-by-side for one provider (which one would you like me to use: ElevenLabs, Cartesia, or Rime?).

from livekit.plugins import ( openai, elevenlabs, deepgram, silero, ) if I am using this?

Thanks — you’re using the LiveKit plugin imports (e.g.,

elevenlabs

) and want to know which parameter shape to use and how those imports relate to LiveKit inference. I’ll keep this short and practical. Quick summary • The imports from

livekit.plugins

give you provider-specific helpers you can call directly (plugin mode). Those calls accept the provider’s native parameters (e.g.,

voice_id

,

voice_settings

for ElevenLabs). • LiveKit Inference is a separate layer that accepts a normalized call shape (e.g.,

model

,

voice

,

language

, plus

extra_kwargs

/

modelOptions

) and forwards those to the provider backend. • If you’re already running inside LiveKit and want a single unified interface across providers, use the LiveKit inference call shape and put any provider-only options inside

extra_kwargs

. If you need direct provider features not exposed by LiveKit’s normalized shape, call the provider plugin directly. When to use each (concrete guidance) • Use plugin call (direct provider parameters) when: ◦ You imported

elevenlabs

,

deepgram

,

silero

, etc. and want to access provider-specific fields or advanced options directly. ◦ You have provider credentials and want provider-native behavior, defaults, or very fine-grained controls (e.g., ElevenLabs

voice_settings.stability

). • Use LiveKit inference shape when: ◦ You want a unified code path across different providers inside LiveKit. ◦ You want LiveKit to handle routing/streaming and just pass provider-specific options in

extra_kwargs

. How to map parameters (examples) 1. Calling provider plugin directly (plugin parameters) • You would pass provider-native fields. Example (ElevenLabs-style parameters): ◦

voice_id

,

voice_settings: {stability, similarity_boost, speed}

,

enable_ssml_parsing

, etc. 1. Calling LiveKit Inference (normalized shape) • Top level:

model

(e.g.,

elevenlabs/eleven_flash_v2_5

),

voice

,

language

• Provider-specific options go in

extra_kwargs

(Node:

modelOptions

). ◦ Example `extra_kwargs`:

{ voice_settings: { stability: 0.5, speed: 1.1 }, enable_ssml_parsing: true }

Which should you choose in your code? • If you imported

elevenlabs

from

livekit.plugins

because you need ElevenLabs-specific options — call the plugin directly with those fields. • If you want to keep the same call shape across providers (so switching from

elevenlabs

to

cartesia

is just changing `model`/`voice`), call LiveKit inference and use

extra_kwargs

for provider specifics. Would you like a short copy-paste example for: • A direct plugin call for ElevenLabs (showing

voice_settings

), or • A LiveKit inference call that uses

model

+

extra_kwargs

for the same ElevenLabs options? Tell me which example you want and I’ll provide it.

show plugin for cartesia , rime .

Thanks — I’ll show you concise, copy-paste plugin examples for Cartesia and Rime (Python), and remind you which plugin parameters to set when you call them directly from

livekit.plugins

. You said you’re using LiveKit imports like: from livekit.plugins import ( openai, elevenlabs, deepgram, silero, ) So below I show the equivalent for Cartesia and Rime and highlight the plugin-specific parameters you’ll pass. Cartesia — plugin usage (Python) 1. Key plugin parameters you can pass: • model (string) — e.g., "sonic-3" (default) • voice (string or list[float]) — voice ID or embedding (default voice id shown in docs) • language (string) — ISO-639-1, e.g., "en" • emotion (string) — e.g., "excited" • speed (float) — speech rate (1.0 = normal) • volume (float) — output volume scaling • api_key (string) — optional override (use env var CARTESIA_API_KEY normally) 1. Example:

from livekit.plugins import cartesia
from livekit.agents import AgentSession

tts = cartesia.TTS(
    model="sonic-3",
    voice="f786b574-daa5-4673-aa0c-cbe3e8534c02",
    language="en",
    emotion="excited",
    speed=1.2,
    volume=1.0
)

session = AgentSession(
    tts=tts,
    # other agent components...
)

Rime — plugin usage (Python) 1. Key plugin parameters you can pass: • model (string) — e.g., "arcana" • speaker (string) — e.g., "celeste" • audio_format (TTSEncoding) — "pcm" or "mp3" • sample_rate (int) — e.g., 16000 • speed_alpha (float) — speech speed control (values < 1.0 are faster) • reduce_latency (bool) — disables some text normalization to reduce latency • phonemize_between_brackets (bool) — allow custom pronunciation in brackets • api_key (string) — optional override (env var RIME_API_KEY normally) 1. Example:

from livekit.plugins import rime
from livekit.agents import AgentSession

tts = rime.TTS(
    model="arcana",
    speaker="celeste",
    audio_format="mp3",
    sample_rate=24000,
    speed_alpha=0.95,
    reduce_latency=True
)

session = AgentSession(
    tts=tts,
    # other agent components...
)

Practical notes • These plugin calls use your own provider credentials (set via the provider-specific env var or pass

api_key

). • Plugin parameter names are provider-native (use them when you call the plugin directly). • If you prefer a unified LiveKit inference call instead, pass

model

,

voice

,

language

at top-level and put provider-specific fields (like

emotion

,

speed

,

voice_settings

) inside

extra_kwargs

(called

modelOptions

in Node.js). • If you want, I can also provide: ◦ A LiveKit inference example that forwards the same Cartesia/Rime options via

extra_kwargs

. ◦ Node.js equivalents for these plugin calls. Which follow-up would help most: the inference examples or Node.js plugin examples? Sources: Cartesia TTS | LiveKit Documentation | Rime TTS | LiveKit Documentation | Text-to-speech (TTS) models overview | LiveKit Documentation

and elevenlabds?

Thanks — I’ll show the ElevenLabs plugin parameters and a concise Python plugin example to match the Cartesia and Rime samples you already have. ElevenLabs — plugin parameters (provider-native) • model (string) — Optional. Default:

eleven_flash_v2_5

• voice_id (string) — Optional. Default example:

EXAVITQu4vr4xnSDxMaL

• voice_settings (object) — Optional. Fields: ◦ stability (float) ◦ similarity_boost (float) ◦ style (float) ◦ use_speaker_boost (bool) ◦ speed (float) • language (string) — Optional. Default:

en

(ISO-639-1) • streaming_latency (int) — Optional. Default:

3

(seconds) • enable_ssml_parsing (bool) — Optional. Default:

false

• chunk_length_schedule (list[int]) — Optional. Default:

[80, 120, 200, 260]

(valid range 50–500) • api_key (string) — Optional override via env var

ELEVEN_API_KEY

Notes on capabilities • Plugin usage gives you full provider-native control (voice cloning/custom voices and detailed

voice_settings

, SSML controls, chunk scheduling, streaming latency). • When using LiveKit Inference instead, you use the normalized shape (

model

,

voice

,

language

, and put provider-specific fields in

extra_kwargs

/

modelOptions

). Use the plugin if you need the native fields directly or custom ElevenLabs voices not available through LiveKit Inference. ElevenLabs — concise Python plugin example

from livekit.plugins import elevenlabs
from livekit.agents import AgentSession

tts = elevenlabs.TTS(
    model="eleven_flash_v2_5",
    voice_id="EXAVITQu4vr4xnSDxMaL",
    language="en",
    streaming_latency=2,
    enable_ssml_parsing=True,
    voice_settings={
        "stability": 0.5,
        "similarity_boost": 0.8,
        "style": 0.2,
        "use_speaker_boost": True,
        "speed": 1.1
    },
    chunk_length_schedule=[100, 150, 250, 300]
)

session = AgentSession(
    tts=tts,
    # other components like stt, llm, etc.
)

If you want, I can next: • Show the equivalent LiveKit inference call that forwards these ElevenLabs options via

extra_kwargs

(useful if you prefer a unified inference path), or • Provide a Node.js plugin example for ElevenLabs. Which would be most helpful? Sources: ElevenLabs TTS | LiveKit Documentation

interruptive adaptive handling in agent 1.4

Thanks — you’re asking about "interruptive adaptive handling" in Agents v1.4. That exact phrase isn’t a documented feature name, but I’ll explain the interrupt and adaptive behaviors that are available in LiveKit Agents 1.4 and how to use them to get the behavior you probably mean (barge‑in, adaptive interruption handling, and resuming). What “interruptive adaptive handling” typically means here • Letting users interrupt (barge in on) agent speech. • Detecting whether the interruption contains speech (a real user intent) or is a spurious/noise interruption. • Adapting agent behavior based on that detection: stop speaking and process user input when valid, optionally resume or repeat when interruption was empty, and allow long-running tools to be cancelled by user speech. Key APIs and controls you’ll use • allow_interruptions flag (in say() / generate_reply()) ◦ Set

allow_interruptions=True

to let user speech cancel ongoing TTS and return control to STT immediately. ◦ Set

allow_interruptions=False

to make speech uninterruptible when you must finish speaking. • SpeechHandle and interruption status ◦ When you call TTS, you can receive a

SpeechHandle

object. ◦ Inspect

speech_handle.interrupted

to check whether a real interruption happened. ◦ Use

speech_handle.cancel()

to stop TTS programmatically if needed. ◦ Long-running tool code should periodically check

speech_handle.interrupted

and cancel early when set. • Turn detection & VAD/endpointing ◦ LiveKit supports model-based turn detection and VAD/endpointing to distinguish real user speech vs. silence/noise. ◦ Use these turn-detection results to decide whether to treat an incoming audio segment as a valid interruption. Practical patterns to implement adaptive interruption handling 1) Barge‑in with validation (recommended) • Start speaking with

allow_interruptions=True

. • When interrupted, use the STT transcript for that interruption: ◦ If transcript contains words (non-empty), treat it as a user turn — stop TTS and handle the user input. ◦ If transcript is empty or only filler/noise, optionally resume speaking or reprompt. 2) Uninterruptible critical messages • For critical prompts or short confirmation messages, call

say(..., allow_interruptions=False)

so the agent completes the message. 3) Cancelling long-running tasks/tools • Start long-running work with access to the current

SpeechHandle

. • Periodically check

speech_handle.interrupted

inside the tool; if true, gracefully cancel the task and return control to the user. 4) Avoiding false interruptions • Combine VAD/endpointing with confidence thresholds from STT. • If VAD triggers but STT confidence is low or transcript is empty, consider it a false interruption and either: ◦ Resume speaking automatically, or ◦ Ask a brief confirmation: “Did you say something?” and only stop when user confirms. Example flow (conceptual) 1. Agent says: say("How can I help you?", allow_interruptions=True) 2. User cuts in: ◦ LiveKit routes audio to STT and produces a transcript. ◦ If transcript non-empty → speech_handle.interrupted becomes True → stop TTS → process transcript as user input. ◦ If transcript empty or very low confidence → treat as false interruption → either resume TTS or re-prompt. Best practices • Use

allow_interruptions=True

for conversational/interactive agents so callers don’t need to wait. • Use short TTS utterances where possible; long monologues increase user frustration if they must wait to interrupt. • Implement a small grace period/confirmation when STT transcript is empty to avoid repeated toggling between speaking and stopping. • Expose a clear UX fallback when interruptions repeatedly fail (e.g., present a visual prompt). • Instrument metrics: counts of interruptions, false interruptions, and cancelled long-running tasks so you can tune thresholds. What I can do next • Provide a concrete code snippet (Python) showing: ◦ using

say(..., allow_interruptions=True)

, ◦ examining

SpeechHandle.interrupted

, and ◦ cancelling a long-running tool when interrupted. • Help you tune thresholds for your use case if you tell me whether your app is noisy (VoIP, conference, phone) or quiet (one-on-one voice). Which would you prefer: a short Python code example demonstrating interruption checks, or a detailed example that includes tool cancellation and resume logic? Sources: Turn Detection for Voice Agents: VAD, Endpointing, and Model-Based Detectio... | Agent speech and audio | LiveKit Documentation | Uninterruptable Agent | LiveKit Documentation | Function tool definition | LiveKit Documentation | Turns overview | LiveKit Documentation, with interactive elements