https://livekit.io logo
Join Slack
Powered by
how to capture metrics for STT, LLM, TTS in liveki...
# ask-ai
e
how to capture metrics for STT, LLM, TTS in livekit agent session? can u give an example code?
t
Thanks for reaching out, Viren. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To capture metrics for STT (Speech-to-Text), LLM (Language Model), and TTS (Text-to-Speech) in a LiveKit agent session, you can use the 
metrics
module and subscribe to the 
metrics_collected
event. Below is an example code snippet:
Copy code
# Import the metrics module
from livekit.agents import metrics

# Create a usage collector to aggregate metrics
usage_collector = metrics.UsageCollector()

# Subscribe to metrics collection events
@agent.on("metrics_collected")
def _on_metrics_collected(mtrcs: metrics.AgentMetrics):
    # Log the metrics using the helper function
    metrics.log_metrics(mtrcs)
    
    # Add metrics to the usage collector for aggregation
    usage_collector.collect(mtrcs)

# Function to log aggregated usage at the end of the session
async def log_usage():
    summary = usage_collector.get_summary()
    <http://logger.info|logger.info>(f"Usage: ${summary}")

# Add the log_usage function as a shutdown callback
ctx.add_shutdown_callback(log_usage)
This code will capture and log metrics for: • STT: Audio duration and connection duration • LLM: Time to first token, input tokens, output tokens, and tokens per second • TTS: Time to first byte and audio duration Let me know if you need further assistance! Sources: Capturing metrics | LiveKit Docs | Capturing metrics | LiveKit Docs
e
i have a session defined as session = AgentSession() and i am running session.start... where should i use the metrics?
t
To use metrics in your session defined as 
session = AgentSession()
, you should set up the metrics collection event handler before calling 
session.start()
. Here’s an example of how to integrate metrics:
Copy code
# Set up metrics collection
usage_collector = metrics.UsageCollector()

@session.on("metrics_collected")
def _on_metrics_collected(ev: MetricsCollectedEvent):
    metrics.log_metrics(ev.metrics)
    usage_collector.collect(ev.metrics)

# Optional: Add a shutdown callback to log usage summary
async def log_usage():
    summary = usage_collector.get_summary()
    <http://logger.info|logger.info>(f"Usage: {summary}")

ctx.add_shutdown_callback(log_usage)

# Then start the session
await session.start(
    agent=MyAgent(),
    room=ctx.room,
    # other options as needed
)
By registering the metrics event handler before starting the session, you can collect and log metrics throughout the session lifecycle. Let me know if you need further clarification! Sources: Capturing metrics | LiveKit Docs | Tree View: | dse-workbench
e
should i set this inside entrypoint function or outside?
t
The metrics setup should be placed inside the entrypoint function. This ensures that the metrics collector and event handlers are properly associated with the specific agent instance and session context. Here’s an example:
Copy code
async def entrypoint(ctx: JobContext):
    # ... other setup code ...
    
    agent = VoicePipelineAgent(
        vad=ctx.proc.userdata["vad"],
        stt=deepgram.STT(),
        llm=openai.LLM(),
        tts=openai.TTS(),
        chat_ctx=initial_ctx,
    )

    usage_collector = metrics.UsageCollector()

    @agent.on("metrics_collected")
    def _on_metrics_collected(mtrcs: metrics.AgentMetrics):
        metrics.log_metrics(mtrcs)
        usage_collector.collect(mtrcs)

    # Start the session or other logic here
By placing the setup inside the entrypoint function, you ensure that it is initialized in the correct context. Let me know if you have further questions! Sources: Tree View: | dse-workbench
12 Views
Open in Slack
PreviousNext
Powered by