I want to start livekit voice ai guide me on installation

Thanks for reaching out, Sajjad. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

yes python

Great — I’ll guide you through installing LiveKit Voice AI with Python. I’ll assume you’re on macOS or Linux unless you tell me otherwise; let me know if you’re on Windows. 1) Prerequisites • Python >= 3.10 installed. • LiveKit CLI installed. • A LiveKit Cloud account (or self-hosted LiveKit credentials). 2) Install LiveKit CLI • macOS:

brew install livekit-cli

• Linux:

curl -sSL https://get.livekit.io/cli | bash

• Windows (if needed): use the Windows package manager tool you prefer. 3) Authenticate the CLI with your LiveKit Cloud account

lk cloud auth

This opens a browser to link your project. 4) Create a Python Voice AI agent from the starter template

lk agent init my-agent --template agent-starter-python
cd my-agent

The CLI clones the template and creates an

.env.local

for credentials. 5) Install Python dependencies The starter uses the

uv

package manager. From the project root:

uv sync

6) Download required model files (VAD, noise cancellation, etc.)

uv run src/agent.py download-files

7) Run the agent in development mode

uv run src/agent.py dev

What you should see: the agent starts and connects to your LiveKit project using the credentials in

.env.local

. The CLI also prints next steps after initialization. If you want, I can: • provide exact commands for Windows, • walk through setting environment variables in

.env.local

, • or help troubleshoot any error you hit while running the steps. Which of those would you like next? , with interactive elements

which version

Thanks — I’ll confirm the versions you need for the Python Voice AI setup. • Python: 3.10 or later (supported up through 3.14). • LiveKit Agents Python SDK: use 1.5.2 or later for full console/debug features; 1.5.0+ recommended for interruption handling. • LiveKit CLI: install the latest CLI release for your OS (no specific minimum version required). • To check the CLI version after install:

lk --version

If you want, I can give the exact

uv

install commands that pin the SDK to a specific version (e.g.,

~=1.5.2

) or walk through verifying your current Python and CLI versions — which would you prefer? Sources: Voice AI quickstart | LiveKit Documentation | Agent Console | LiveKit Documentation | Build a multilingual voice agent that automatically switches languages | Li... | Build a multilingual voice agent that automatically switches languages | Li... | Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit, with interactive elements

"livekit-agents[silero,turn-detector]~=1.3", "livekit-plugins-google>=1.5.8", "livekit-plugins-noise-cancellation~=0.2", "python-dotenv>=1.2.2", create requirements.txt with latest version

Thanks — I’ll create a requirements.txt for the Python Voice AI setup using the latest compatible versions. Save this as requirements.txt:

livekit-agents[silero,turn-detector]~=1.5
livekit-plugins-google~=1.5
livekit-plugins-noise-cancellation~=0.2
python-dotenv>=1.2.2

Would you like the pip install command for these, or a pinned-version file instead? Sources: Why is my Agent insights data missing? | LiveKit | Why is my Agent insights data missing? | LiveKit, with interactive elements

is it necesary uv add torch

Short answer: No — you usually don't need to run

uv add torch

manually. •

livekit-agents[silero,...]

pulls in the Silero models which declare

torch

as a dependency, so installing the package will install a compatible CPU

torch

by default. • Install

torch

yourself only if you need a specific build (e.g., GPU/CUDA support) or a pinned version. In that case, install the appropriate

torch

wheel before or after

uv add

using the official PyTorch instructions for your CUDA/Python combo. If you want, tell me your OS, Python version, and whether you need GPU/CUDA — I’ll give the exact `pip`/`uv` command to install the correct

torch

build. , with interactive elements

how to remove torch

You're asking how to remove

torch

. Which environment are you using: a system Python, a virtualenv/venv, or the

uv

project environment? Also let me know if you installed via `pip`/`uv` or a system package manager. Common commands: • In a virtualenv/venv or system Python (pip): ◦ Uninstall:

pip uninstall torch

• Confirm removal:

pip show torch || echo "torch not installed"

• In a

uv

project: a. Remove from the project:

uv remove torch

1. Sync to apply changes:

uv sync

1. Verify:

uv list | grep torch || echo "torch not present"

• If

torch

is pinned in your

requirements.txt

or `pyproject`: ◦ Edit

requirements.txt

and remove the

torch...

line, or remove the dependency entry in your project config, then reinstall/sync. Notes: • If you installed a GPU/CUDA wheel manually (pip with a wheel URL), uninstall with

pip uninstall torch

and then reinstall the desired build if needed. • If other packages depend on

torch

, uninstall may fail or leave those packages broken—check

pipdeptree

or

uv list

to see dependents. Tell me which environment and install method you used and I’ll provide the exact commands. , with interactive elements

athar@athar-computer:~/python_projects/WORKSHOP_DEMO/livekit-voice-agent$ uv run agent.py download-files INFOlivekit.agentsDownloading files for livekit.plugins.silero 134705.048 INFO livekit.agents Downloading files for livekit.plugins.silero INFOlivekit.agentsFinished downloading files for livekit.plugins.silero 134705.049 INFO livekit.agents Finished downloading files for livekit.plugins.silero INFOlivekit.agentsDownloading files for livekit.plugins.turn_detector 134705.050 INFO livekit.agents Downloading files for livekit.plugins.turn_detector [transformers] PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. WARNINGhuggingface hub.utils. httpwarning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. 134706.366 WARNING huggingf…ils._http Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. INFOlivekit.agentsFinished downloading files for livekit.plugins.turn_detector 134709.446 INFO livekit.agents Finished downloading files for livekit.plugins.turn_detector

Thanks — I see the log: the agent download step completed but printed the Hugging Face warning and this line: "[transformers] PyTorch was not found. Models won't be available..." — that means models that require Torch were not downloaded because PyTorch is missing. What you can do next (pick one): 1) Install CPU-only PyTorch (quickest)

pip install torch --index-url https://download.pytorch.org/whl/cpu

2) Install GPU/CUDA PyTorch (if you have NVIDIA GPU) — choose the right CUDA version from PyTorch site, e.g. for CUDA 11.8:

pip install torch --index-url https://download.pytorch.org/whl/cu118

3) If you're using

uv

project environment:

uv add torch
uv sync

Or install the desired wheel with pip inside the uv environment before

uv sync

. After installing Torch, re-run the download:

uv run agent.py download-files

Optional: set HF_TOKEN to avoid Hugging Face rate limits:

export HF_TOKEN=<your_token>

(or add to

.env.local

) Tell me your OS, Python version, and whether you need GPU/CUDA and I’ll give the exact torch wheel command. , with interactive elements

why install torch

Good question — short answer: torch is required only if you need to run PyTorch-based models locally (Silero VAD, some turn-detection or HF transformer models). The download step warned because those models need PyTorch to load weights; without torch you get tokenizers/configs but not runnable models. When you need torch: • You want the agent to run models locally (VAD, speech models, or HF transformer models). • You want faster inference or offline usage without calling external model hosts. When you don't need torch: • You plan to use remote/model-hosted inference (e.g., cloud APIs) or only use components that don't require PyTorch. In that case the agent can still run but some features will be unavailable. Do you want CPU-only or GPU (CUDA) support? Tell me your OS, Python version, and whether you have an NVIDIA GPU and I’ll give the exact install command (pip/uv) and next steps. , with interactive elements

is it not downloaded when i run uv run agent.py download-files

Good question — the download step only fetches model files and assets, not the PyTorch library itself. •

uv run agent.py download-files

downloads plugin model files (weights, configs). Some of those files require PyTorch to actually load and use the models. • The warning "[transformers] PyTorch was not found. Models won't be available..." means the files were downloaded (or metadata was fetched) but you can't load/run the PyTorch models until

torch

is installed. Next steps: • If you want to run models locally, install torch (choose CPU or GPU/CUDA): ◦ CPU-only:

pip install torch --index-url https://download.pytorch.org/whl/cpu

• GPU (example for CUDA 11.8):

pip install torch --index-url https://download.pytorch.org/whl/cu118

• Or in a uv project:

uv add torch
    uv sync

• After installing torch, re-run:

uv run agent.py download-files

• If you don't need local PyTorch (you'll use remote/model-hosted inference), you can ignore the warning — models that require PyTorch won't be usable locally. Tell me your OS, Python version, and whether you need GPU/CUDA and I’ll give the exact install command for your environment. , with interactive elements

multilingula models

You're asking about multilingual models — I’ll summarize what’s available and how to enable them. • Supported multilingual turn-detection model: LiveKit provides a MultilingualModel that supports Chinese, Dutch, English, French, German, Indonesian, Italian, Japanese, Korean, Portuguese, Russian, Spanish, and Turkish. It can handle mixed-language conversations and runs efficiently on CPU. • How to enable multilingual turn detection and STT in Python:

from livekit.plugins.turn_detector.multilingual import MultilingualModel
from livekit.agents import AgentSession, inference, TurnHandlingOptions

session = AgentSession(
    turn_handling=TurnHandlingOptions(
        turn_detection=MultilingualModel(),
    ),
    stt=inference.STT(language="multi"),
)

• Torch requirement: The multilingual turn detector (and some STT/voice components) use model weights that require PyTorch to load. If you want to run these models locally, install

torch

(CPU or GPU build). If you use remote/model-hosted inference, local

torch

is not required. • Download step: Run

uv run agent.py download-files

to fetch model files; install

torch

and re-run the download if you need runnable PyTorch models. If you want, tell me: • whether you need local CPU or GPU inference, and • whether you plan to use hosted inference (LiveKit Inference) or local models — and I’ll give the exact install commands and a requirements.txt tuned for multilingual usage. Sources: Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Build a healthcare intake assistant with an Anam avatar | LiveKit | LiveKit’s Series B: Building the all-in-one platform for voice AI agents | ... | LiveKit turn detector plugin | LiveKit Documentation, with interactive elements

gentActivity._pipeline_reply_task_impl.<locals>._on_first_frame() at /home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_activity.py:2532 handle: <Handle AgentActivity._pipeline_reply_task_impl.<locals>._on_first_frame() at /home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_activity.py:2532> Traceback (most recent call last): File "/home/athar/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/events.py", line 89, in _run self._context.run(self._callback, *self._args) ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_activity.py", line 2563, in _on_first_frame self._session._update_agent_state( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ "speaking", ^^^^^^^^^^^ start_time=started_speaking_at, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ otel_context=speech_handle._agent_turn_context, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_session.py", line 1520, in _update_agent_state self.emit( ~~~~~~~~~^ "agent_state_changed", ^^^^^^^^^^^^^^^^^^^^^^ AgentStateChangedEvent(old_state=old_state, new_state=state), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_session.py", line 482, in emit super().emit(event, arg) ~~~~~~~~~~~~^^^^^^^^^^^^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/rtc/event_emitter.py", line 58, in emit callback(*callback_args) ~~~~~~~~^^^^^^^^^^^^^^^^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/agent.py", line 116, in _on_agent_state_changed elapsed = time.time() - last_eou_metrics.timestamp() 142105.210 ERROR asyncio Exception in callback AgentActivity._pipeline_reply_task_impl.<locals>._on_first_frame() at /home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_activity.py:… handle: <Handle AgentActivity._pipeline_reply_task_impl.<locals>._on_first_frame() at /home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_activity.py:… Traceback (most recent call last): File "/home/athar/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/events.py", line 89, in _run self._context.run(self._callback, *self._args) ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_activity.py", line 2563, in _on_first_frame self._session._update_agent_state( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ "speaking", ^^^^^^^^^^^ start_time=started_speaking_at, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ otel_context=speech_handle._agent_turn_context, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_session.py", line 1520, in _update_agent_state self.emit( ~~~~~~~~~^ "agent_state_changed", ^^^^^^^^^^^^^^^^^^^^^^ AgentStateChangedEvent(old_state=old_state, new_state=state), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/voice/agent_session.py", line 482, in emit super().emit(event, arg) ~~~~~~~~~~~~^^^^^^^^^^^^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/rtc/event_emitter.py", line 58, in emit callback(*callback_args) ~~~~~~~~^^^^^^^^^^^^^^^^ File "/home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/agent.py", line 116, in _on_agent_state_changed elapsed = time.time() - last_eou_metrics.timestamp() ~~~~~~~~~~~~~~~~~~~~~~~~~~^^

Thanks — I looked at the traceback you pasted. It shows the exception occurs here in your

agent.py

callback: elapsed = time.time() - last_eou_metrics.timestamp() That indicates

last_eou_metrics

is None (or otherwise missing

timestamp()

), raised when the session emits

agent_state_changed

. Fix by guarding access and logging when the metric is absent. Suggested quick fix (in your

agent.py

_on_agent_state_changed

handler):

def _on_agent_state_changed(self, event):
    try:
        if last_eou_metrics is None:
            # handle missing metrics (first event or not yet emitted)
            logger.debug("last_eou_metrics not set; skipping elapsed calc")
            elapsed = None
        else:
            elapsed = time.time() - last_eou_metrics.timestamp()
        # continue with your logic using `elapsed` (check for None)
    except Exception:
        logger.exception("error in agent_state_changed handler")

Notes and next checks: • Ensure whatever sets

last_eou_metrics

runs before you use it (log when you set it). • If

last_eou_metrics

comes from the session/event arg, read it from the event rather than a global variable. • Add defensive logging to confirm the metric is emitted before the first

speaking

state. If you share the relevant lines around where

last_eou_metrics

is set and your

_on_agent_state_changed

implementation, I’ll suggest a minimal patch tailored to your code. , with interactive elements

@session.on("agent_state_changed") def _on_agent_state_changed(ev: AgentStateChangedEvent): if ( ev.new_state == "speaking" and last_eou_metrics and session.current_speech # and last_eou_metrics.speech_id == session.current_speech.id and getattr(last_eou_metrics, "speech_id", None) == session.current_speech.id and getattr(last_eou_metrics, "last_speaking_time", None) is not None ): delta = ev.created_at - last_eou_metrics.last_speaking_time logger.info("Time to first audio frame: %sms", delta.total_seconds() * 1000) the logs never get printed

Thanks — I’m looking at your

@session.on("agent_state_changed")

handler and why the log never prints. Quick checklist and concrete steps to debug: 1) Confirm the event fires • Add an unconditional debug log at the top to ensure the callback runs:

logger.debug("agent_state_changed fired: %s -> %s", ev.old_state, ev.new_state)

2) Verify each condition • Log the values used in the if so you can see which check fails:

logger.debug("conds: new_state=%s last_eou=%s current_speech=%s speech_id_match=%s last_speaking_time=%s",
             ev.new_state, bool(last_eou_metrics), bool(session.current_speech),
             getattr(last_eou_metrics, "speech_id", None) if last_eou_metrics else None,
             getattr(last_eou_metrics, "last_speaking_time", None) if last_eou_metrics else None)

3) Common reasons your block never runs •

last_eou_metrics

is None or not yet set when the event fires. •

session.current_speech

is None or its

id

doesn’t match

last_eou_metrics.speech_id

. •

last_eou_metrics.last_speaking_time

is None or not a datetime. •

ev.new_state

value differs (e.g., capitalization or different string like

"speaking_changed"

). 4) Defensive fix (avoid silent misses) • Compute and early-return with logs so you still get info:

@session.on("agent_state_changed")
def _on_agent_state_changed(ev: AgentStateChangedEvent):
    logger.debug("agent_state_changed fired: %s -> %s", ev.old_state, ev.new_state)
    if ev.new_state != "speaking":
        return

    if not last_eou_metrics:
        logger.debug("last_eou_metrics missing")
        return

    if not session.current_speech:
        logger.debug("session.current_speech missing")
        return

    if getattr(last_eou_metrics, "speech_id", None) != session.current_speech.id:
        logger.debug("speech_id mismatch: %s != %s", getattr(last_eou_metrics, "speech_id", None), session.current_speech.id)
        return

    if getattr(last_eou_metrics, "last_speaking_time", None) is None:
        logger.debug("last_speaking_time missing")
        return

    delta = ev.created_at - last_eou_metrics.last_speaking_time
    logger.info("Time to first audio frame: %sms", delta.total_seconds() * 1000)

5) Check types and time origin • Ensure

ev.created_at

and

last_eou_metrics.last_speaking_time

are both timezone-aware datetimes or both naive. Mixing them can raise errors or produce unexpected results. 6) If logs still don’t appear • Confirm your logger level and handler configuration allow DEBUG/INFO messages to stdout/file. • Confirm no exception is raised before the log (wrap handler body in try/except and log exceptions). If you share: • where/how

last_eou_metrics

is set, • sample prints from the debug logs above, I’ll pinpoint the failing check and propose the minimal code change. , with interactive elements

(livekit-voice-agent) athar@athar-computer:~/python_projects/WORKSHOP_DEMO/livekit-voice-agent$ Agents Starting console mode 🚀 DEBUGasyncioUsing selector: EpollSelector 142927.039 DEBUG asyncio Using selector: EpollSelector INFOlivekit.agentsstarting worker 142927.041 INFO livekit.agents starting worker {"version": "1.5.8", "rtc-version": "1.1.7"} INFOlivekit.agentsplugin registered 142927.042 INFO livekit.agents plugin registered {"plugin": "livekit.plugins.silero", "version": "1.5.8"} INFOlivekit.agentsplugin registered 142927.043 INFO livekit.agents plugin registered {"plugin": "livekit.plugins.turn_detector.base", "version": "1.5.8"} INFOlivekit.agentspreloading plugins INFO livekit.agents preloading plugins {"packages": ["livekit.plugins.silero", "livekit.plugins.turn_detector", "av"]} INFOlivekit.agentsstarting inference executor 142927.046 INFO livekit.agents starting inference executor INFOlivekit.agentsinitializing process 142928.006 INFO livekit.agents initializing process {"pid": 30221, "inference": true} INFOlivekit.agentsprocess initialized 142932.418 INFO livekit.agents process initialized {"pid": 30221, "inference": true, "elapsed_time": 4.41} INFOlivekit.agentsHTTP server listening on :45879 142932.419 INFO livekit.agents HTTP server listening on :45879 WARNINGlivekit.agentsno warmed process available for job, waiting for one to be created 142932.421 WARNING livekit.agents no warmed process available for job, waiting for one to be created {"job_id": "mock-job-24d983bacfd3"} INFOlivekit.agentsinitializing job runner 142932.422 INFO livekit.agents initializing job runner {"tid": 30249} DEBUGasyncioUsing selector: EpollSelector 142932.423 DEBUG asyncio Using selector: EpollSelector INFOlivekit.agentsjob runner initialized 142932.424 INFO livekit.agents job runner initialized {"tid": 30249, "elapsed_time": 0.0} WARNINGlivekit.agentspreemptive_generation is deprecated and will be removed in v2.0. Use turn_handling=TurnHandlingOptions(...) instead 142932.646 WARNING livekit.agents preemptive_generation is deprecated and will be removed in v2.0. Use turn_handling=TurnHandlingOptions(...) instead /home/athar/python_projects/WORKSHOP_DEMO/livekit-voice-agent/agent.py75 DeprecationWarning: UsageCollector is deprecated. Use ModelUsageCollector from metrics.model_usage instead. usage_collector = metrics.UsageCollector() WARNINGlivekit.agentsmetrics_collected is deprecated. Use session_usage_updated for usage tracking and ChatMessage.metrics for per-turn latency. 142932.647 WARNING livekit.agents metrics_collected is deprecated. Use session_usage_updated for usage tracking and ChatMessage.metrics for per-turn latency. INFOlivekit.agentsadaptive interruption detector initialized 142932.649 INFO livekit.agents adaptive interruption detector initialized {"base_url": "https://agent-gateway.livekit.cloud/v1", "detection_interval": 0.1, "audio_prefix_duration": 1.0, "max_audio_duration": 3, "min_frames": 2, "threshold": 0.5, "inference_timeout": 0.7, "use_proxy": true} DEBUGlivekit.agentshttp_session(): creating a new httpclient ctx 142932.653 DEBUG livekit.agents http_session(): creating a new httpclient ctx DEBUG main agent_state_changed fired: initializing -> listening 142932.660 DEBUG main agent_state_changed fired: initializing -> listening DEBUGlivekit.agentsusing audio io:

Console

->

AgentSession

->

TranscriptSynchronizer

->

Console

142932.662 DEBUG livekit.agents using audio io:

Console

->

AgentSession

->

TranscriptSynchronizer

->

Console

DEBUGlivekit.agentsusing transcript io:

AgentSession

->

TranscriptSynchronizer

142932.663 DEBUG livekit.agents using transcript io:

AgentSession

->

TranscriptSynchronizer

default ▇ ▇ █ ▇ ▆ ▅ ▅ ▅ ▄ ▃ ▁ ▁ ▁ ▁ 142937.007 INFO livekit.agents STT metrics {"model_name": "assemblyai/universal-streaming", "model_provider": "livekit", "audio_duration": 4.15} default ▇ ▆ ▄ █ ▆ ▅ ▄ ▅ ▄ ▃ ▁ ▁ ▁ ▁ 142937.009 DEBUG livekit.agents received user transcript {"user_transcript": "Hello. How are you?", "language": "en", "transcript_delay": 1.2043633460998535} default ▅ ▆ ▄ █ ▅ ▅ ▅ ▄ ▄ ▃ ▁ ▁ ▁ ▁ 142937.256 DEBUG livekit.…_detector eou prediction {"eou_probability": 0.2777174711227417, "duration": 0.024, "input": "<|im_start|>user\nhello how are you"} default ▅ ▆ ▄ █ ▅ ▅ ▅ ▄ ▄ ▃ ▁ ▁ ▁ ▁ 142937.261 DEBUG livekit.agents using preemptive generation {"preemptive_lead_time": 0.25055527687072754} default ▅ ▆ ▆ █ ▅ ▄ ▅ ▄ ▅ ▄ ▁ ▁ ▁ ▁ 142937.263 INFO livekit.agents EOU metrics {"model_name": "multilingual", "model_provider": "livekit", "end_of_utterance_delay": 1.45, "transcription_delay": 1.21} default ▅ ▆ ▆ █ ▅ ▄ ▅ ▄ ▅ ▄ ▁ ▁ ▁ ▁ 142937.269 DEBUG main agent_state_changed fired: listening -> thinking default █ █ ▇ █ ▆ ▅ ▄ ▄ ▄ ▃ ▁ ▁ ▁ ▁ 142940.113 INFO livekit.agents LLM metrics {"model_name": "google/gemini-2.5-flash", "model_provider": "livekit", "ttft": 2.81, "prompt_tokens": 35, "prompt_cached_tokens": 0, "completion_tokens": 24, "tokens_per_second": 7.74} default █ ▆ ▇ ▇ ▆ ▅ ▄ ▄ ▄ ▄ ▁ ▁ ▁ ▁ 142940.338 DEBUG livekit.agents aec warmup active, disabling interruptions for 3.00s default █ ▆ ▇ ▇ ▆ ▅ ▄ ▄ ▄ ▄ ▁ ▁ ▁ ▁ 142940.340 DEBUG main agent_state_changed fired: thinking -> speaking default █ ▆ ▇ ▇ ▆ ▅ ▄ ▄ ▄ ▄ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142940.341 DEBUG main last_speaking_time missing default ▅ ▅ ▄ ▅ ▅ ▅ ▇ █ ▇ ▆ ▄ ▄ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142941.823 INFO livekit.agents TTS metrics {"model_name": "cartesia/sonic-3", "model_provider": "livekit", "ttfb": 0.367171538000548, "audio_duration": 3.72} default ▄ ▄ ▄ █ ▆ ▅ ▄ ▅ ▄ ▄ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142943.338 DEBUG livekit.agents aec warmup expired, re-enabling interruptions default ▃ ▆ ▇ █ ▇ ▅ ▇ ▆ ▅ ▄ ▄ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142944.434 DEBUG main agent_state_changed fired: speaking -> listening default █ ▆ ▅ ▇ ▆ ▄ ▅ ▄ ▄ ▃ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142949.409 INFO livekit.agents STT metrics {"model_name": "assemblyai/universal-streaming", "model_provider": "livekit", "audio_duration": 9.5} default █ ▆ ▅ ▇ ▆ ▄ ▅ ▄ ▄ ▃ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142949.412 DEBUG livekit.agents received user transcript {"user_transcript": "What are you saying?", "language": "en", "transcript_delay": 0.5506985187530518} default ▆ ▆ ▅ █ ▇ ▅ ▄ ▃ ▄ ▃ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142949.487 DEBUG livekit.…_detector eou prediction {"eou_probability": 0.2566692531108856, "duration": 0.069, "input": "<|im_start|>user\nhello how are you<|im_end|>\n<|im_start|>assistant\nalright alright settle down let's get to the tech support what's wrong with your computer<|im_end|>\n<|im_start|>user\nwhat are you saying"} default ▆ ▆ ▅ █ ▇ ▅ ▄ ▃ ▄ ▃ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142949.490 DEBUG livekit.agents using preemptive generation {"preemptive_lead_time": 0.07607650756835938} default ▆ ▆ ▅ █ ▇ ▅ ▄ ▃ ▄ ▃ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142949.492 INFO livekit.agents EOU metrics {"model_name": "multilingual", "model_provider": "livekit", "end_of_utterance_delay": 0.63, "transcription_delay": 0.55} default ▆ ▆ ▅ █ ▇ ▅ ▄ ▃ ▄ ▃ ▁ ▁ ▁ ▁ end_of_turn: 1453ms · turn_completed_cb: 0.00ms · llm_ttft: 2813ms · tts_ttfb: 478ms ─ e2e: 4533ms 142949.494 DEBUG main agent_state_changed fired: listening -> thinking default ▇ ▅ ▆ █ ▆ ▅ ▅ ▄ ▄ ▄ ▁ ▁ ▁ ▁ 142951.789 INFO livekit.agents LLM metrics {"model_name": "google/gemini-2.5-flash", "model_provider": "livekit", "ttft": 2.16, "prompt_tokens": 64, "prompt_cached_tokens": 0, "completion_tokens": 28, "tokens_per_second": 11.8} default ▅ ▅ ▄ █ ▆ ▄ ▄ ▄ ▅ ▃ ▁ ▁ ▁ ▁ 142952.114 DEBUG main agent_state_changed fired: thinking -> speaking default ▅ ▅ ▃ █ ▆ ▅ ▄ ▄ ▄ ▄ ▁ ▁ ▁ ▁ end_of_turn: 629ms · turn_completed_cb: 0.00ms · llm_ttft: 2160ms · tts_ttfb: 507ms ─ e2e: 3253ms 142952.116 DEBUG main last_speaking_time missing default ▇ ▇ ▄ ▇ ▆ ▆ ▇ █ ▇ ▇ ▅ ▂ ▁ ▁ end_of_turn: 629ms · turn_completed_cb: 0.00ms · llm_ttft: 2160ms · tts_ttfb: 507ms ─ e2e: 3253ms 142953.439 INFO livekit.agents TTS metrics {"model_name": "cartesia/sonic-3", "model_provider": "livekit", "ttfb": 0.3496470190002583, "audio_duration": 4.13} default ▆ ▆ ▅ █ ▆ ▆ ▅ ▅ ▅ ▄ ▂ ▁ ▁ ▁ end_of_turn: 629ms · turn_completed_cb: 0.00ms · llm_ttft: 2160ms · tts_ttfb: 507ms ─ e2e: 3253ms 142956.639 DEBUG main agent_state_changed fired: speaking -> listening default ▅ ▅ ▇ █ ▇ ▅ ▄ ▄ ▅ ▃ ▁ ▁ ▁ ▁ end_of_turn: 629ms · turn_completed_cb: 0.00ms · llm_ttft: 2160ms · tts_ttfb: 507ms ─ e2e: 3253ms 142959.338 INFO livekit.agents STT metrics {"model_name": "assemblyai/universal-streaming", "model_provider": "livekit", "audio_duration": 9.8} de

Thanks — I reviewed your logs. The handler is firing (you see many "agent_state_changed fired" lines) but your block never runs because

last_eou_metrics

is missing when the session enters

speaking

. The log "last_speaking_time missing" confirms that. Quick steps to fix and diagnose: 1. Ensure

last_eou_metrics

is set from the EOU metrics event. Add a handler to capture and store it:

@session.on("eou_metrics")
def _on_eou_metrics(ev):
    global last_eou_metrics
    last_eou_metrics = ev  # or copy the relevant fields (speech_id, last_speaking_time)
    logger.debug("saved last_eou_metrics: %s", getattr(last_eou_metrics, "last_speaking_time", None))

1. Make the state handler defensive and verbose so you can see which condition fails:

@session.on("agent_state_changed")
def _on_agent_state_changed(ev: AgentStateChangedEvent):
    logger.debug("agent_state_changed fired: %s -> %s", ev.old_state, ev.new_state)
    logger.debug("last_eou_metrics=%s current_speech=%s", bool(last_eou_metrics), bool(session.current_speech))
    if ev.new_state != "speaking":
        return
    if not last_eou_metrics:
        logger.debug("last_eou_metrics missing")
        return
    if not session.current_speech:
        logger.debug("session.current_speech missing")
        return
    if getattr(last_eou_metrics, "speech_id", None) != session.current_speech.id:
        logger.debug("speech_id mismatch: %s != %s", getattr(last_eou_metrics, "speech_id", None), session.current_speech.id)
        return
    if getattr(last_eou_metrics, "last_speaking_time", None) is None:
        logger.debug("last_speaking_time missing")
        return

    delta = ev.created_at - last_eou_metrics.last_speaking_time
    logger.info("Time to first audio frame: %sms", delta.total_seconds() * 1000)

1. Check types/timezones • Verify

ev.created_at

and

last_eou_metrics.last_speaking_time

are both datetimes of the same kind (both timezone-aware or both naive). 1. Logging level • Ensure your logger prints DEBUG/INFO to console (it appears to, given your output). If you prefer, share the code that sets

last_eou_metrics

(or the

eou_metrics

event payload) and I’ll propose an exact small patch to ensure it’s stored and matched to the correct

speech_id

. , with interactive elements

@session.on("metrics_collected") def _on_metrics_collected(ev: MetricsCollectedEvent): nonlocal last_eou_metrics if ev.metrics.type == "eou_metrics": last_eou_metrics = ev.metrics metrics.log_metrics(ev.metrics) usage_collector.collect(ev.metrics)

Thanks — you provided the

metrics_collected

handler. The issue is likely one of timing, scope, or event type: your handler sets

last_eou_metrics

only when

ev.metrics.type == "eou_metrics"

, but the

agent_state_changed

to "speaking" can occur before that assignment runs (or the variable isn't in the expected scope), so your checks see it as missing. Do these quick fixes: 1) Use the dedicated EOU event (more reliable) and save a minimal struct

@session.on("eou_metrics")
def _on_eou_metrics(ev):
    nonlocal last_eou_metrics
    last_eou_metrics = types.SimpleNamespace(
        speech_id=getattr(ev, "speech_id", None),
        last_speaking_time=getattr(ev, "last_speaking_time", None),
    )
    logger.debug("saved last_eou_metrics: %s", last_eou_metrics)

2) Make your metrics_collected handler defensive (optional)

@session.on("metrics_collected")
def _on_metrics_collected(ev: MetricsCollectedEvent):
    nonlocal last_eou_metrics
    if getattr(ev.metrics, "type", None) == "eou_metrics":
        last_eou_metrics = ev.metrics
    metrics.log_metrics(ev.metrics)
    usage_collector.collect(ev.metrics)

3) Harden the state handler (already suggested — ensure you log each failing condition) • Keep the detailed debug logs you added so you can see whether last_eou_metrics exists, whether speech_id matches, and whether last_speaking_time is present. 4) Check variable scope • Ensure

last_eou_metrics

is declared in the surrounding scope and modified with

nonlocal

(or

global

) consistently in all handlers. 5) Verify timestamps/types • Confirm

last_eou_metrics.last_speaking_time

and

ev.created_at

are both datetimes of the same kind (both timezone-aware or both naive). If needed, convert to UTC before subtracting. If you want, paste the

eou_metrics

payload (what gets logged by your debug line) and I’ll give an exact minimal patch to store the right fields and match

speech_id

reliably. , with interactive elements

install livekit cli

Got it — here are concise install steps for the LiveKit CLI for macOS, Linux, and Windows, plus verification and auth. macOS (Homebrew) 1. Install:

brew install livekit-cli

1. Verify:

lk --version

Linux (script) 1. Install:

curl -sSL https://get.livekit.io/cli | bash

1. Restart your shell (or add ~/.livekit/bin to PATH), then verify:

lk --version

Windows (winget) 1. Install:

winget install LiveKit.LiveKitCLI

1. Verify in PowerShell:

lk --version

Authenticate with your LiveKit Cloud account 1. Run:

lk cloud auth

This opens a browser to sign in and link a project. Notes • If you prefer a manual binary, download releases from the LiveKit CLI releases page (choose the OS binary) and add it to your PATH. • Tell me your OS if you want exact PATH fixes or if you hit errors during install., with interactive elements