https://livekit.io logo
Join Slack
Powered by
What's the best way to measure the latency from th...
# ask-ai
b
What's the best way to measure the latency from the moment the user finishes speaking to the moment the agent first makes a sound?
✅ 1
t
Thanks for reaching out, Amir. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’ll answer your question about measuring latency from when the user stops speaking to when the agent first makes a sound. Short answer • Use the per-turn end-to-end latency metric (
e2e_latency
) which measures the time between the end of the user utterance and the agent’s verbal response onset. • For deeper diagnosis, break the total into stages: end-of-turn detection, LLM time-to-first-token (TTFT), and TTS time-to-first-byte (TTFB). How to measure (two options) • Observability dashboard (quick, visual) ◦ Open Agent Observability to view synchronized audio, transcripts, and traces. ◦ Use the timeline to click the end of the user utterance and the agent audio onset — the UI shows the exact timestamps and computed e2e latency. • Programmatic / metrics access (precise, automatable) a. Record per-turn metrics for each ChatMessage and read the 
e2e_latency
field. b. If you need bottleneck detail, also collect: • 
end_of_turn_delay
(user finished-speaking detection) • 
llm_node_ttft
(LLM time-to-first-token) • 
tts_node_ttfb
(TTS time-to-first-byte) a. Sum or compare those fields to get the total and per-stage latencies. Suggested targets to evaluate performance • Aim for total e2e latency < 800 ms • Individual goals: STT <200 ms, LLM TTFT <300 ms, TTS TTFB <300 ms Would you like help locating the 
e2e_latency
field in your data or a short snippet for reading these metrics programmatically? Sources: Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Understand and Improve Agent Latency | LiveKit | Understand and Improve Agent Latency | LiveKit, with interactive elements
b
where on the livekit dashboard is observability ?
t
Thanks — I’m responding to your question about where to find Observability in the LiveKit dashboard. • Open your project in the LiveKit Cloud dashboard. • In the session list, open the session you want to inspect. • Click the "Agent insights" tab for that session — Agent Observability lives there (Transcript, Trace, and Logs views). If you don’t see "Agent insights," can you tell me whether Observability is enabled in your project settings under Data and privacy, or share the project ID so I can guide you next? Sources: Streamline troubleshooting with Agent Observability | LiveKit | Why is my Agent insights data missing? | LiveKit | Why is my Agent insights data missing? | LiveKit | LiveKit Agent Observability | Debug and optimize voice agents | LiveKit, with interactive elements
b
should STT+EOU+LLM TTFT +TTS TTFB equal to the e2e_latency?
t
Short answer: usually yes — e2e_latency should approximately equal the sum of STT + end-of-utterance (EOU) delay + LLM TTFT + TTS TTFB, but there are caveats. Details: • e2e_latency is the measured time from the end of the user’s utterance to the agent’s verbal response onset, so it covers those pipeline stages. • Differences can occur because: ◦ Some stages overlap (e.g., partial transcription or early LLM streaming) so sums of individual stage durations may double-count or miss overlaps. ◦ Measurement boundaries and rounding between components can introduce small discrepancies. ◦ Network or queuing delays outside those measured fields can add extra time not attributed to a single stage. • Use the Trace view in Agent insights to see exact timestamps for each node (STT, EOU, LLM, TTS) and confirm how they line up for a given turn. If you want, share one example turn’s trace (or the metric values) and I’ll check whether the numbers add up and point out any overlaps or extra delays. , with interactive elements
b
For livekit function tool calls, is there a built in way of getting the agent to say a filler phrase such as 'one moment please' while it is doing the function call?
t
Thanks — regarding your question about filler phrases during function tool calls: • There is no built‑in, automatic filler-phrase feature for function tool calls. • You can implement it easily in your tool by calling 
context.session.say()
at the start of the function to play a phrase like “one moment please.” • For lower latency or consistent audio, pre-synthesize and cache the filler TTS at startup and play that audio during the call. You can also use 
BackgroundAudioPlayer
to play ambient “thinking” sounds. Would you like a short code snippet (Python or JS) showing how to add 
context.session.say()
inside a function tool? Sources: The ReAct Pattern for Voice Agents and How AI Agents Think, Act, and Respon... | Audio customization | LiveKit Documentation | External data and RAG | LiveKit Documentation, with interactive elements
3 Views
Open in Slack
PreviousNext
Powered by