We have livekit working in production with STT deephram LLM LiveKit Community #ask-ai

We have livekit working in production with STT (de...

chilly-mechanic-10995

08/04/2025, 3:13 PM

We have livekit working in production with STT (deephram), LLM (GPT 4.1) and TTS (Elevenlabs 2.5 Turbo) models. But as soon we try to change the model (LLM to Groq Llama 4 and TTS to Rime AI) in agent.py, we are not getting all the eou, stt, tts metrics to check the latency in logs. All the logs are coming empty. Is there a configuration in agent.py that we missing? Please check in detail and answer this

tall-airline-16756

08/04/2025, 3:13 PM

Thanks for reaching out, Ashish. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

tall-airline-16756

08/04/2025, 3:13 PM

When changing LLM and TTS models in

agent.py

, ensure the following configurations are in place to capture metrics like EOU, STT, and TTS latency: 1. Subscribe to Metrics Events: Add a subscription to the

metrics_collected

event to capture all relevant metrics. For example:

Copy code

@agent.on("metrics_collected")
def _on_metrics_collected(mtrcs: metrics.AgentMetrics):
    metrics.log_metrics(mtrcs)

2. Implement Specific Metric Handlers: For metrics like EOU, STT, and TTS, ensure their respective events are handled. For example, for EOU metrics:

Copy code

@agent.stt.on("eou_metrics_collected")
async def handle_eou_metrics(metrics: EOUMetrics):
    <http://logger.info|logger.info>(f"EOU Delay: {metrics.end_of_utterance_delay}")

3. Aggregate Metrics (Optional): Use

UsageCollector

to aggregate metrics across sessions for comprehensive logging:

Copy code

usage_collector = metrics.UsageCollector()

@agent.on("metrics_collected")
def _on_metrics_collected(mtrcs: metrics.AgentMetrics):
    usage_collector.collect(mtrcs)

async def log_usage():
    summary = usage_collector.get_summary()
    <http://logger.info|logger.info>(f"Session Usage: {summary}")

ctx.add_shutdown_callback(log_usage)

Ensure these configurations are correctly implemented to resolve the issue of empty logs. Let me know if you need further assistance! Sources: Capturing metrics | LiveKit Docs | metrics/metrics_stt.py

icy-cpu-74166

08/04/2025, 6:10 PM

The code that you have provided was already in use and was working fine. When the LLM model changed to Groq Llama 4 and TTS to Rime AI, the LLM and TTS calls are working fine and the agent responses are coming as expected. Even in the metrics, events are getting for all STT, EOU, LLM, and TTS but there is no data in it (like ttft, ttfb, tokens) etc. Note: 1. We are passing api_key from the groq.LLM(), rime.TTS() methods and not setting them as Environment variables/ 2. Even though there is no model change for STT, vad or Turn Detector, the EOU and STT metric logs are getting printed with null values. Please clarify the root cause of this.

tall-airline-16756

08/04/2025, 6:10 PM

Hey there! It looks like the reason why metrics like

ttft

ttfb

, and

tokens

are showing up as null when using Groq Llama 4 and Rime AI could be due to a few things: 1. Provider Support: Sometimes, Groq Llama 4 and Rime AI might not support or report these specific metrics. It’s a good idea to double-check if their APIs actually provide timing metrics like

ttft

and

ttfb

, as well as token counts. 2. Integration Code: Make sure the integration code for these providers is set up properly to report the needed metrics. If the data isn’t sent in the right format, those fields will stay null. 3. Metric Logging Configuration: The

log_metrics

function depends on the data from the LLM and TTS providers. If that data is incomplete or not formatted right, the metrics won’t get logged correctly. 4. Metric Aggregation: If you’re using

UsageCollector

to gather metrics, missing or incomplete data from the providers can lead to null values in the final results. Here are some friendly steps to help you troubleshoot: 1. Take a look at the documentation for Groq Llama 4 and Rime AI to confirm they support these metrics and are set up to provide them. 2. Review your integration code to make sure it’s handling and reporting all the required metrics properly. 3. Try turning on verbose logging for the metrics module—that way, you can catch any errors or warnings about missing metrics. 4. Check if there are any known issues or limitations with LiveKit’s integration for these providers. If you want, I’m here to help you walk through any of these steps—just let me know! Sources: livekit-agents/livekit/agents/metrics/base.py | Capturing metrics | LiveKit Docs | agents/src/metrics/base.ts

icy-cpu-74166

08/04/2025, 6:15 PM

Note: We are passing api_key from the groq.LLM(), rime.TTS() methods and not setting them as Environment variables. Can this cause any issue?

tall-airline-16756

08/04/2025, 6:15 PM

Hey! You can definitely pass API keys directly in method calls like

groq.LLM()

and

rime.TTS()

instead of setting them as environment variables, and it usually won’t mess with metrics logging. That said, using environment variables is generally a safer bet for security. If you’re still having trouble with metrics logging, it’s probably not because of how the API keys are passed. It might be a good idea to double-check your integration code and make sure metrics are properly enabled and configured. Feel free to reach out if you want me to help you troubleshoot further! Sources: livekit-plugins/livekit-plugins-rime/livekit/plugins/rime/tts.py | Deploying to production | LiveKit Docs

icy-cpu-74166

08/04/2025, 6:23 PM

For your previous reply, you mentioned that there could some issues with the providers of Groq, Rime but after making this change, I'm not even getting any data for STT and EOU as well where I haven't changed any models of VAD, STT and turn detector. how come this is happening then?

tall-airline-16756

08/04/2025, 6:23 PM

Hey there! When you switch up your LLM and TTS models, even if the VAD, STT, and turn detector models stay the same, a few things might cause the STT and EOU metrics to stop showing data: 1. Event Listeners: Make sure the event listeners for collecting metrics are properly reattached after you change the models. Without these, metrics like STT and EOU won’t be captured. 2. Turn Detection Compatibility: Double-check that your new LLM model plays nicely with the current turn detection mode. For those EOU metrics to come through, turn detection needs to be set up correctly, especially if you’re using VAD or server-side detection. 3. Model Capabilities: Confirm that your new LLM and TTS models support sending the necessary metrics events. Some models handle events differently, which can affect how metrics are reported. If you want, I’m here to help with reattaching event listeners or checking model compatibility—just let me know!

Open in Slack

Previous Next