is it possible to set other codecs when using twilio & livekit? the audio quality is quite bad. We use

PCMU/8000

according to the Livekit info in the telephony section

Thanks for reaching out, Marc. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Do you control the SIP endpoint that Twilio connects to (so we can check whether Opus/wideband is negotiable)?

yes, Twilio sends a request to a service I control, then we use TwiML to dial like

<?xml version="1.0" encoding="UTF-8"?><Response><Dial><Sip>sip:+19...@5q....sip.livekit.cloud;transport=tcp</Sip></Dial><Hangup /></Response>

Thanks — this is helpful. You control the SIP endpoint and are using TwiML to dial into LiveKit, so we can try to negotiate better codecs on the SIP/WebRTC boundary and apply LiveKit audio processing to improve what room participants hear. Here’s a concise plan and actionable steps you can take. 1) Verify and advertise Opus in your SIP/SDP handling • When Twilio sends the INVITE to your service, ensure the SIP INVITE/SDP you forward or respond with advertises Opus and wideband (16 kHz+) in the media codecs list. If LiveKit sees Opus offered/acceptable, it will use Opus for the WebRTC leg to room participants. • Ensure your SIP response (or the SDP you pass to LiveKit) includes

opus/48000

in the m=audio line and the relevant RTPmap/rtpmap/rtcp-fb attributes so Opus is a selectable codec. 2) Ensure LiveKit telephony endpoint accepts/transcodes correctly • LiveKit will accept PCMU from Twilio and can transcode to Opus for WebRTC clients, but it will pick Opus if it’s offered and negotiated. So make sure LiveKit’s SIP URI and the SIP handling on your service present Opus support in SDP to LiveKit. 3) Configure Twilio to offer or allow wideband codecs • In your Twilio SIP trunk settings or the TwiML flow, include/allow Opus (and other wideband codecs) so Twilio can advertise them to your service if Twilio supports doing so for that call type. If Twilio’s PSTN endpoint forces PCMU, the SIP leg to your service may still be limited — check the actual INVITE/SDP Twilio sends to you to confirm which codecs it offers. 4) Inspect actual SDP flow (diagnostics) • Capture the SIP INVITE SDP Twilio sends to your service and the SDP your service sends to LiveKit. Compare codec lists and the final negotiated codec. Key checks: ◦ Does the Twilio INVITE list

opus/48000

? ◦ Does your forwarded/answered SDP contain

opus/48000

in the m=audio line? ◦ What codec does LiveKit ultimately select for the WebRTC RTP stream? • If you paste anonymized SDP snippets here I can help read them and point out exact changes. 5) Turn on LiveKit audio enhancements • Enable noise cancellation / telephony audio processing (Krisp or BVCTelephony) in LiveKit for the room or agent clients. That reduces noise and improves perceived quality even when the inbound codec is PCMU. 6) Fallback realities and expectations • If Twilio or the PSTN endpoint is constrained to PCMU on the SIP side, you cannot change the PSTN source codec. But LiveKit will transcode and apply processing so the audio delivered to WebRTC participants uses Opus and sounds much better than raw G.711 narrowband in most cases. If you want, share the following and I’ll give exact SDP edits and TwiML/SIP response examples: • The SDP from the Twilio INVITE (anonymized). • The SDP your service currently forwards or responds with to LiveKit (anonymized). • Whether you control the SIP proxy that forwards Twilio’s SIP INVITE to LiveKit (so I know whether to suggest modifying the forwarded INVITE or the 200/183 response). I can also provide a sample SIP/SDP snippet and a small TwiML/SIP-handling example that explicitly advertises Opus and wideband audio. Which would you like next?

ok so if both Opus and PCMU/8000 are offered, Livekit will choose the best codec? do I need to do anything in livekit?

Thanks — I’m responding to your question about whether LiveKit will pick Opus when both Opus and PCMU/8000 are offered, and whether you need to change anything in LiveKit. Short answer • Yes. When both Opus and PCMU/8000 are offered, LiveKit will prefer Opus for the WebRTC leg. You typically do not need to change LiveKit itself — just ensure Opus is advertised in the SDP offered to LiveKit and enable audio processing if you want additional quality improvements. What to check / do • Ensure the SDP Twilio sends or the SDP your service forwards to LiveKit includes

opus/48000

in the audio m= line and the appropriate rtpmap/rtcp-fb attributes so Opus is a selectable codec. • Inspect the actual SDP exchange (Twilio -> your service -> LiveKit) to confirm Opus is present. If Twilio offers Opus and LiveKit sees it, LiveKit will select Opus for the WebRTC participants. • Optionally enable LiveKit’s noise cancellation / telephony processing (Krisp or BVCTelephony) to improve perceived quality even when the inbound SIP/PSTN codec is PCMU. What you do not need to do • You don’t need to change LiveKit’s default behavior to make it prefer Opus — it already prefers higher-quality codecs when available. If you want, I can: • Read anonymized SDP snippets (Twilio INVITE and the SDP you forward to LiveKit) and point out exactly where to add/confirm Opus. • Provide an example SDP reply or TwiML/SIP snippet that explicitly advertises Opus. Which would help most right now: an SDP check or an example snippet?

Optionally enable LiveKit’s noise cancellation / telephony processing (Krisp or BVCTelephony) to improve perceived quality even when the inbound SIP/PSTN codec is PCMU.

I use

noise_cancellation.BVCTelephony()

in the code like this

audio_input=AudioInputOptions(
                noise_cancellation=get_noise_cancellation_model(ctx),
            ),

and also Krisp checked in the SIP trunk. Is it correct? Should be just one of them?

Thanks — I’m responding to your question about using

noise_cancellation.BVCTelephony()

in code while also having Krisp enabled on the SIP trunk. Short answer • Use only one noise cancellation method at a time. Running both

BVCTelephony()

in the agent code and Krisp on the SIP trunk can make audio worse. I recommend using

BVCTelephony()

for telephony/SIP calls and disabling Krisp on the trunk. Why only one • Noise cancellation models expect raw audio. If you apply one NC stage first, the second stage receives already-processed audio, which can: ◦ Reduce STT accuracy ◦ Remove or distort speech (including the first words) ◦ Let through background conversations or create artifacts Why

BVCTelephony()

is the recommended choice for your setup •

BVCTelephony()

is optimized for narrowband telephony audio (typical 8 kHz phone calls) and for voice-AI use cases where you want the primary speaker preserved and background speakers filtered out. That improves STT and downstream voice processing compared with stacking multiple NCs. What to do now (clear next steps) 1. Keep your current code:

audio_input=AudioInputOptions(
       noise_cancellation=get_noise_cancellation_model(ctx),
   ),

where

get_noise_cancellation_model(ctx)

returns

noise_cancellation.BVCTelephony()

. 1. Disable Krisp in the Twilio SIP trunk settings (

krisp_enabled=false

). 2. Test with a few calls and verify: ◦ STT accuracy ◦ Clarity (especially first words) ◦ Background noise suppression If you’d prefer Krisp instead of BVCTelephony • Remove the

noise_cancellation

option from your agent code and keep

krisp_enabled=true

on the SIP trunk. Then run the same tests above to compare. If you want, I can help with: • A quick checklist to compare call samples (what to listen for / metrics to check). • Interpreting test results or recommending which one to keep after you run trials. Which would you like next? Sources: Choosing the right noise cancellation for your agent | LiveKit

I'm confused. In here: https://docs.livekit.io/transport/media/noise-cancellation/ it says

Krisp noise cancellation can be applied directly at your SIP trunk for inbound or outbound calls. This uses the standard Krisp noise cancellation (NC) model. Other models are not available for SIP.

in the telephony section. But there is a BVCTelephony model, and BVC is supposed to be better than NC. So I'm confused

Thanks — I’m responding to your confusion about the docs saying “Krisp is available on SIP trunks” while you also see a

BVCTelephony

model that seems better. Both statements are correct; they just apply to different places in the call path. I’ll clarify how they differ and give a clear recommendation. Why the docs look contradictory • Krisp on the SIP trunk: Krisp’s standard noise cancellation model can be applied directly at the SIP trunk (the SIP/RTP leg). That means the incoming PSTN/SIP audio from Twilio can be processed by Krisp before it reaches your LiveKit endpoint. •

BVCTelephony()

in LiveKit: This is a LiveKit-side noise cancellation model run inside the LiveKit audio pipeline (on the WebRTC leg / agent client). It’s a BVC model optimized for telephony (narrowband) and voice‑AI use cases. So: Krisp runs on the SIP trunk; BVCTelephony runs inside LiveKit. The doc line that “Other models are not available for SIP” refers to what you can run at the SIP trunk itself — only Krisp (standard NC) is available there. Which one to use (recommended) • Use only one noise cancellation stage. Running both creates double-processing that harms STT and audio quality. • For telephony/SIP calls, I recommend using

BVCTelephony()

inside LiveKit and disabling Krisp on the SIP trunk. ◦ Why:

BVCTelephony()

is designed for narrowband telephony audio and for voice-AI scenarios where you want to keep the primary speaker clean while filtering background talkers. It improves STT accuracy and preserves first words. • Alternative: If you can’t change LiveKit client code or prefer trunk-side processing, enable Krisp on the SIP trunk and remove LiveKit’s noise_cancellation setting. This is acceptable but typically not as good for voice-AI/stt scenarios. Clear actionable steps 1. Keep your agent code using:

audio_input=AudioInputOptions(
       noise_cancellation=get_noise_cancellation_model(ctx),
   ),

where

get_noise_cancellation_model(ctx)

returns

noise_cancellation.BVCTelephony()

. 1. Disable Krisp on your Twilio/SIP trunk (

krisp_enabled=false

). 2. Run test calls and check: ◦ STT accuracy ◦ Clarity (especially first words) ◦ Background suppression 1. If needed, try the opposite (disable BVCTelephony in code and enable Krisp on the trunk) to compare, but keep only one active. If you want, send a short sample of results or describe what you hear (STT errors, missing first words, artifacts). I can give a quick checklist of what to listen for and help interpret the comparison.

ok and what model do you suggest to use given my pipeline Inbound call from PSTN → Twilio number → My IVR service -> dial to LK SIP Trunk -> my agent Idk if better

deepgram/nova-3:en

or

deepgram/nova-2-phonecall

Thanks — I’m responding to your question about which STT model to use for your pipeline: PSTN → Twilio → IVR → LiveKit SIP trunk → Agent Recommendation (short) • Use deepgram/nova-3:en. Why (concise) • Nova-3 outperforms Nova-2 Phonecall in accuracy and speed across audio types, including telephony. Nova-2 Phonecall is effectively legacy/phone-optimized but Nova-3 is newer and generally better even on 8 kHz calls. • Nova-3 gives better future compatibility and improvements, and supports multilingual modes if you need them later. How to apply it • Set the model in your AgentSession (or equivalent STT config) like:

session = AgentSession(
    stt="deepgram/nova-3:en",
    # ...
)

• Keep using

BVCTelephony()

for noise cancellation on the LiveKit side (disable Krisp on the SIP trunk) as we discussed—this combination plus Nova-3 should give the best STT and audio quality for your pipeline. Testing tips • Run A/B tests with short call samples (same audio) transcribed by: ◦

deepgram/nova-3:en

+ BVCTelephony ◦ (optional)

deepgram/nova-2-phonecall

+ BVCTelephony (for comparison) • Evaluate: ◦ Word error rate / obvious transcription errors ◦ STT handling of initial words and overlapping speech ◦ Latency to first transcript segment If you want, I can: • Provide a short checklist for A/B testing and metrics to capture. • Help tune STT params (e.g., punctuation, profanity filters, language setting) for better results. Which would you like next? Sources: Building Performant Voice Agents in India | LiveKit | Deepgram STT | LiveKit Documentation

so something like this? inference.STT( model="deepgram/nova-3:en", extra_kwargs=dict(keyterm=KEYTERMS), ),

Thanks — I’m responding to your example using

inference.STT

with

deepgram/nova-3:en

. Yes — that looks correct. A few quick confirmations and tips: 1. Example usage • Your snippet is fine:

inference.STT(
    model="deepgram/nova-3:en",
    extra_kwargs=dict(keyterm=KEYTERMS),
)

• Ensure

KEYTERMS

is the structure the STT expects (e.g., a list of strings or whatever your STT integration requires). 1. Things to verify • Make sure your STT config is actually wired into your AgentSession or pipeline so audio from the LiveKit agent is routed to this STT instance. • If you want multi-language support later, consider

language=multi

or the appropriate model tag for Nova-3. • Tune any model-specific options (punctuation, profanity, timestamps) via

extra_kwargs

as needed. 1. Testing suggestion • Run a short A/B test with the same call audio to confirm improved WER and first-word capture when using

nova-3

+

BVCTelephony()

(with Krisp disabled). • Capture transcripts, latency to first transcript, and common error types to validate improvements. If you’d like, paste the exact

KEYTERMS

structure you plan to use and I’ll confirm it’s formatted correctly for the STT call, or I can suggest extra_kwargs options to improve accuracy for your use case.

but do I write :en or use the language kwarg

Thanks — I’m responding to whether to put the language in the model string (":en") or use a

language

kwarg. Short answer • Using the model string with the language suffix (e.g.,

"deepgram/nova-3:en"

) is the recommended and simplest approach. • You can use a separate

language

kwarg for multi-language or explicit overrides, but don’t specify both in conflicting ways. Details and guidance • Model suffix approach: ◦ Example:

model="deepgram/nova-3:en"

explicitly selects the English Nova-3 model. This is clear and concise and is what you used in your snippet. • Language kwarg approach: ◦ Use

language="multi"

(or

language="en"

) when you want the STT call to be language-aware separately from the model identifier, or when your API accepts a

language

parameter for runtime language detection/overrides. • Do not combine conflicting settings: ◦ If you include the

:en

suffix, avoid also setting

language="es"

(or other conflicting values). That can create ambiguity about which setting takes precedence. Examples • Simple (explicit model-language):

inference.STT(
    model="deepgram/nova-3:en",
    extra_kwargs=dict(keyterm=KEYTERMS),
)

• Explicit model + language override (only use if your STT integration expects it and they agree):

inference.STT(
    model="deepgram/nova-3:en",
    language="multi",  # only when you intentionally want multi-language handling
    extra_kwargs=dict(keyterm=KEYTERMS),
)

When to choose which • If your calls are reliably English, use

model="deepgram/nova-3:en"

. • If you expect multiple languages on the same call or want the service to auto-detect, use

language="multi"

(and choose a model that supports multi-language if needed). If you want, share the exact STT client/library signature you’re using and I’ll confirm the exact parameter names and a final example tailored to your code.

i'm using this

from livekit.agents import inference, stt, tts

Thanks — I’m responding to your last two points: you’re importing

from livekit.agents import inference, stt, tts

and asking how to specify the language (":en" vs language kwarg) when using the LiveKit Agents STT. Short answer • Import

inference

from

livekit.agents

(don’t import

stt

or

tts

as separate modules). • Use the model string with the language suffix (e.g.,

"deepgram/nova-3:en"

) for simplicity when your calls are reliably English. You may also pass

language="multi"

or

language="en"

as a separate kwarg if your code path supports it, but avoid conflicting settings. What to change in your code 1. Correct import • Use:

from livekit.agents import AgentSession, inference

• Do not import

stt

or

tts

directly as modules (those are not the documented import pattern). 1. STT config examples • Model-string approach (recommended for fixed-language English):

session = AgentSession(
    stt="deepgram/nova-3:en",
    # ...
)

• Alternatively, explicit inference.STT call:

session = AgentSession(
    stt=inference.STT(
        model="deepgram/nova-3:en",
        extra_kwargs=dict(keyterm=KEYTERMS),
    ),
    # ...
)

• If you need multi-language detection:

session = AgentSession(
    stt=inference.STT(
        model="deepgram/nova-3",   # or "deepgram/nova-3:en" plus language override if supported
        language="multi",
        extra_kwargs=dict(keyterm=KEYTERMS),
    ),
)

Note: Don’t set conflicting language values (e.g.,

model="deepgram/nova-3:en"

and

language="es"

). Quick checklist before testing • Ensure this STT object is actually wired into your AgentSession so audio from LiveKit’s pipeline goes to the STT. • Keep

noise_cancellation.BVCTelephony()

enabled on the LiveKit side and disable Krisp on the SIP trunk (only one NC stage). • Run short test calls and compare transcripts/latency to confirm improvements. If you want, paste your current AgentSession code (anonymized) and I’ll return a corrected snippet you can drop into your service. Sources: Agent session | LiveKit Documentation | Deepgram STT | LiveKit Documentation

what about flux for deepgram?

Thanks — I’m responding to your question about “flux for Deepgram” and how it fits into your PSTN → Twilio → IVR → LiveKit SIP trunk → agent pipeline. Short answer • Deepgram Flux is an STT model optimized for conversational turn-based audio with built‑in turn detection. Its LiveKit model name is

deepgram/flux-general

. Use Flux when you need high-quality, low-latency turn detection and conversational behavior in English. For multilingual or highest-accuracy general transcription, use

deepgram/nova-3:en

(or Nova-3 multi) instead. When to choose Flux vs Nova-3 • Choose Flux (deepgram/flux-general) when: ◦ Your calls are English-only. ◦ Turn detection (knowing when a speaker ends) is important for agent handoffs, live dialogue control, or low-latency end‑of‑turn decisions. ◦ You want a model optimized specifically for conversational call flows with built-in endpointing. • Choose Nova-3 (deepgram/nova-3:en) when: ◦ You want the best overall transcription accuracy across varied audio types (including multi-language options). ◦ You need multilingual support or the absolute best WER for content beyond conversational turn-handling. Key facts for your pipeline • Flux is English-only; Nova-3 supports multilingual modes. • Flux offers built-in turn detection; you can set LiveKit’s

turn_detection="stt"

to consume that behavior. • Flux is available via LiveKit Inference (simple to use) or via the Deepgram plugin if you want to use your own Deepgram account and tune parameters like eager EOT thresholds. • Pricing: Flux is mid-tier (cheaper than Nova-3 monolingual, pricier than older models) — consider cost vs need for turn detection. How to use Flux in LiveKit Agents (example) • Using LiveKit Inference (simplest):

from livekit.agents import AgentSession, inference

session = AgentSession(
    stt=inference.STT(
        model="deepgram/flux-general",
        language="en"
    ),
    turn_detection="stt",
    # ... other config (tts, llm, vad, etc.)
)

• Using Deepgram plugin (if you want custom Deepgram params / your account):

from livekit.plugins import deepgram

stt = deepgram.STTv2(
    model="flux-general-en",
    eager_eot_threshold=0.4,  # plugin-specific tuning
)
session = AgentSession(stt=stt, turn_detection="stt")

Notes and recommendations for your setup • If your primary need is accurate STT for agent actions and you also need explicit turn/endpoint detection, try Flux first (with

turn_detection="stt"

). It reduces the need for separate EOT heuristics. • If you need best-in-class accuracy across languages or content types, use Nova-3 and implement VAD/turn-detection separately (e.g., VAD + server-side heuristics or LiveKit turn detection). • Keep BVCTelephony enabled in LiveKit and disable Krisp on the SIP trunk (only one NC stage). • Run a short A/B: same call audio through Flux vs Nova-3 to compare WER, latency to first transcript, and turn detection reliability. If you want, I can: • Provide a test checklist and metrics to capture for an A/B comparison. • Give a ready-to-drop AgentSession snippet tailored to your current code and noise-cancellation choices. Which would help most? Sources: Deepgram STT | LiveKit Documentation | Speech-to-text (STT) models overview | LiveKit Documentation | Build voice, video, and physical AI | LiveKit | Deepgram STT plugin guide | LiveKit Documentation | HIPAA Eligible Products and Services | LiveKit

• ok so if I use flux I need

turn_detection="stt"

but for nova I would keep

from livekit.plugins.turn_detector.english import EnglishModel

? How do I ensure the correct

turn_detection

is used? I have this

return stt.FallbackAdapter(
        [
            inference.STT(
                model="flux-general-en",
                keyterm=KEYTERMS,
            ),
            inference.STT(
                model="deepgram/nova-3:en",
                extra_kwargs=dict(keyterm=KEYTERMS),
            ),
            deepgram.STT(
                model="nova-3:en",
                extra_kwargs=dict(keyterm=KEYTERMS),
            ),
            inference.STT(
                model="deepgram/nova-2-phonecall",
                language="en",
                # the intensifier can be ommited,
                # see <https://github.com/livekit/agents/issues/4633>
                extra_kwargs=dict(keywords=KEYTERMS),
            ),
            inference.STT(
                model="assemblyai",
            ),
        ]
)

and the AgentSession(stt=that, turn_detection=EnglishModel) • I don't need multi-lingual, english only • primary use case is phone calls, i.e. streaming

Thanks — I’m responding to your question about which

turn_detection

to use when you have Flux first in a

FallbackAdapter

and Nova as fallbacks, for an English-only, telephony streaming pipeline. Short answer • Use

turn_detection="stt"

for the session if Flux (

deepgram/flux-general

/

flux-general-en

) is your primary STT. Flux includes built‑in semantic endpointing and works with

turn_detection="stt"

. • Do not set

turn_detection

to

EnglishModel

when you rely on Flux —

turn_detection

is a single session-level setting and won’t automatically switch per STT in the fallback list. Why •

turn_detection

is global for the AgentSession; it is not dynamically switched based on which STT in a FallbackAdapter is used. • Flux provides its own high-quality turn detection. Setting

turn_detection="stt"

lets LiveKit use the STT provider’s endpointing (Flux). • Nova models do not provide the same built-in semantic turn detection. If Flux fails over to Nova, the STT-based endpointing won’t produce the same EOT behavior — but that’s a fallback scenario. In normal operation Flux (first in list) drives turn decisions. Recommended config for your pipeline (PSTN → Twilio → IVR → LiveKit SIP trunk → Agent) • Put Flux first in your fallback list. • Use

turn_detection="stt"

. • Keep a lightweight VAD (e.g., Silero) for interruption handling. • Keep BVCTelephony enabled (disable Krisp on SIP trunk). Example pattern (drop-in)

from livekit.agents import AgentSession, inference
from livekit.plugins import silero, deepgram  # if using plugin fallbacks

stt_chain = stt.FallbackAdapter([
    inference.STT(model="flux-general-en", extra_kwargs=dict(keyterm=KEYTERMS)),
    inference.STT(model="deepgram/nova-3:en", extra_kwargs=dict(keyterm=KEYTERMS)),
    deepgram.STT(model="nova-3:en", extra_kwargs=dict(keyterm=KEYTERMS)),
    inference.STT(model="deepgram/nova-2-phonecall", language="en", extra_kwargs=dict(keywords=KEYTERMS)),
    inference.STT(model="assemblyai"),
])

session = AgentSession(
    stt=stt_chain,
    turn_detection="stt",      # Use STT-based endpointing (Flux provides this)
    vad=silero.VAD.load(),     # Keep VAD for interruption/latency handling
    # ... other config
)

Notes and practical tips • Flux-first ensures normal calls use Flux’s endpointing and low-latency turn detection. If Flux cannot be used and the session falls back to Nova, expect less-effective STT-driven EOT — handle that as a degraded fallback case. • You do not need

EnglishModel

when using Flux. It’s redundant and adds latency. • Tune Flux endpointing parameters if exposed (e.g., eager EOT thresholds) if you get false positives/negatives. • Run A/B tests: same call samples via ◦ Flux + turn_detection="stt" + BVCTelephony ◦ Nova-3 + external turn detector (if you want to compare) Capture: latency to first transcript, correct EOT detection, STT word errors, and STT interruptions. If you’d like, I can: • Produce a ready-to-run AgentSession snippet matching your exact imports and types. • Give a short checklist of metrics and a sample test harness to compare Flux vs Nova-3 in your environment. Which would you prefer? Sources: Deepgram STT plugin guide | LiveKit Documentation | Deepgram STT | LiveKit Documentation | Improved End-of-Turn Model Cuts Voice AI Interruptions 39% | Updating the Turn Detector Model | LiveKit | AssemblyAI plugin guide | LiveKit Documentation

but is flux STT in a STT-LLM-TTS pipeline, or a realtime model?

Thanks — I’m responding to whether Deepgram Flux is suitable as a realtime/streaming STT model in an STT → LLM → TTS pipeline, or if it’s only a non-realtime/batch model. Short answer • Flux is a real‑time streaming STT model designed for conversational, turn‑based audio. It’s intended for low‑latency pipelines like STT → LLM → TTS where you need fast transcripts and reliable end‑of‑turn detection. Key details • Real-time streaming: Flux is optimized for live conversational streams and provides low latency so you can feed incremental transcripts into an LLM and produce TTS responses quickly. • Built‑in turn detection: Flux includes acoustic + semantic endpointing (end‑of‑turn) which is why you should use

turn_detection="stt"

when Flux is your primary STT — it signals reliably when a speaker finished, which is critical for interactive agent flows. • Best for English conversational flows: Flux is English-only and is tuned for conversational/phone-call style audio (turns, interruptions, natural pauses). • Integration options: ◦ Use Flux via LiveKit Inference (simplest) to get streaming transcripts and STT-based endpointing with no extra Deepgram setup. ◦ Or use the Deepgram plugin (your Deepgram account) for additional tuning (e.g., eager EOT thresholds) if you need fine control. • Fallback behavior: If Flux is first in a FallbackAdapter, normal operation will be real-time with Flux handling endpointing. If Flux fails and you fall back to Nova-3 (which lacks Flux’s semantic endpointing), turn detection will degrade unless you supply another turn detector (but note turn_detection is a session-level setting and won’t switch automatically per STT model). Practical recommendations for an STT→LLM→TTS realtime pipeline • Put Flux first in your STT fallback chain if you want the built-in endpointing to drive conversation flow. • Set

turn_detection="stt"

on the AgentSession to consume Flux’s endpointing. • Keep a low-latency VAD (e.g., Silero) to help with interruption handling. • Keep BVCTelephony noise cancellation enabled in LiveKit (disable Krisp on the SIP trunk). • If you need multilingual support or the absolute best transcription across diverse audio, test Nova‑3; but for English conversational phone calls where turn detection and latency matter, Flux is the right choice. If you want, I can: • Provide a minimal streaming AgentSession example wired to an LLM and TTS stage to show how transcripts flow into the LLM and trigger TTS on end-of-turn. • Give a short checklist to measure latency, EOT accuracy, and transcription quality for Flux vs Nova‑3 in your environment. Which would you like next?