I am having these metrics, are these good?
LLM Per...
# ask-aif
future-stone-69754
06/27/2025, 5:33 AMI am having these metrics, are these good?
LLM Performance:
• Average response time: 1.51 seconds
• Average tokens per second: 14.17
TTS Performance:
• Average time to first byte: 0.54 seconds
STT Performance:
• Average transcription delay: 0.57 seconds
• Average end-of-utterance detection: 0.58 seconds
t
tall-airline-16756
06/27/2025, 5:33 AMThanks for reaching out, Baymurat. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
06/27/2025, 5:33 AMThe metrics you provided are useful for evaluating performance, but I couldn't find specific benchmarks or guidelines to determine if these values are considered "good." Performance standards can vary depending on the use case, system requirements, and user expectations. If you have specific goals or benchmarks in mind, comparing your metrics against those would be the best approach. Let me know if you'd like assistance with further analysis or setting benchmarks.
f
future-stone-69754
06/27/2025, 5:33 AMhow can I improve the performance?
t
tall-airline-16756
06/27/2025, 5:34 AMHey there! Want to boost the performance of your LLM, TTS, and STT metrics? I've got some friendly tips that can really help you out:
• Use the Agents metrics API to track and analyze performance metrics for LLM, STT, TTS, and VAD (Voice Activity Detection). It's like having a performance health check for your services!
• Get strategic about network proximity. Think of it like placing your agent in the perfect spot to communicate smoothly with LLM, STT, and TTS services. Low network latency is your friend here. Each provider might have their own optimization secrets, so peek at their docs.
• Keep an eye on those key latency metrics - Time To First Token (TTFT) and Time To First Byte (TTFB). These are like the speedometers of your system's performance.
I've also got some awesome example implementations that can help you track these metrics in the LiveKit Python agents examples repository:
• LLM metrics
• STT metrics
• TTS metrics
• VAD metrics
Need more help? I'm all ears! Just give me a shout, and we'll dive deeper together. Sources: How can I reduce latency in voice agents using STT, TTS and LLM? | LiveKit | Recipes and examples for voice AI and more. | LiveKit Docs