https://livekit.io logo
Join Slack
Powered by
how to get usage metrics summary through new event...
# ask-ai
a
how to get usage metrics summary through new event when call ends
t
Thanks for reaching out, Harshita. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m responding about how to get a usage metrics summary when a call ends. Short answer: • LiveKit does not emit a built-in “call-ended usage summary” event. • You can produce one by aggregating the 
metrics_collected
events during the session and sending a final summary when the session ends (or using the Data Collection mode payload and adding metrics yourself). How to implement (high level): 1. Subscribe to metrics events during the session: ◦ Listen for 
AgentSessionEventTypes.MetricsCollected
(or your runtime’s equivalent) and accumulate relevant fields (e.g., input/output tokens, inference times, audio durations). ◦ Example pattern:
Copy code
session.on(voice.AgentSessionEventTypes.MetricsCollected,
                lambda ev: usage_collector.collect(ev.metrics))
1. Detect session end: ◦ Use the session/room end event or the Agent Builder Data Collection end webhook (the job/room 
ended_at
payload). 1. Emit/send the summary at end: ◦ On session end, compute aggregates (totals, averages) from your collector and either: ▪︎ POST the summary to your endpoint, or ▪︎ Emit a custom event into your system or room for downstream consumers. 1. Optional: include summary in Data Collection mode ◦ If you use Data Collection mode, add your computed usage metrics into the final structured payload (
summary
or 
results
) before sending it to your configured endpoint. If you’d like, tell me: • Which SDK/language you’re using, and • Whether you prefer the summary sent via webhook or emitted in-room, and I’ll provide a short code example tailored to that environment. Sources: Agent Console | LiveKit Documentation | Collect structured data with LiveKit Agents | LiveKit | Agent Builder | LiveKit Documentation | Creating and Publishing Custom Metrics, with interactive elements
a
when to use ModelUsageCollector, 
LLMModelUsage
, 
TTSModelUsage
, 
STTModelUsage, / session.usage.model_model_usage
t
Thanks — you’re asking when to use 
ModelUsageCollector
, 
LLMModelUsage
, 
TTSModelUsage
, 
STTModelUsage
, and 
session.usage.model_model_usage
. Short, practical guidance: • ModelUsageCollector ◦ Use this to aggregate metrics throughout a session. Subscribe to 
metrics_collected
events and let the collector sum token counts, audio durations, and timings so you can produce a final summary at call end. • 
LLMModelUsage
◦ Use per-inference when you need granular LLM data (input/output token counts, per-call cost/latency). Good for debugging or per-turn accounting. • 
TTSModelUsage
◦ Use per-inference to capture generated audio metrics (duration, characters). Useful when you need per-synthesis cost, quality analysis, or to attribute audio output to a step. • 
STTModelUsage
◦ Use per-inference to capture speech-to-text metrics (audio duration transcribed, confidence metadata). Good for per-segment transcription accounting and troubleshooting. • 
session.usage.model_model_usage
◦ Read this for session-level totals once aggregated. Use it for billing reports, dashboards, or adding a usage summary to your end-of-call payload. If you want, tell me which SDK/language you’re using and whether you prefer the final summary pushed as a webhook or emitted in-room — I’ll give a short code example. Sources: Adaptive interruption handling | LiveKit Documentation | Agent Console | LiveKit Documentation, with interactive elements
a
which one is deprecated and which one to use at the end of the call to log usage
t
Thanks — quick answer about deprecation and what to use at call end. • None of the pieces you listed are deprecated. ◦ 
LLMModelUsage
, 
TTSModelUsage
, and 
STTModelUsage
are the per-inference usage types. ◦ 
ModelUsageCollector
(or 
UsageCollector
) is the helper to aggregate metrics during a session. ◦ The supported session-level property to read at end-of-call is 
session.usage.model_usage
. • Best practice to log usage at call end: a. Aggregate metrics during the session (subscribe to 
metrics_collected
and use `ModelUsageCollector`/`UsageCollector`). b. On session shutdown/room end, read 
session.usage.model_usage
and log or POST those records. If you tell me your SDK/language, I’ll provide a short code snippet for subscribing, collecting, and logging at session end. Sources: Build a multilingual voice agent that automatically switches languages | Li..., with interactive elements
a
InterruptionMetrics: { detectionDelay: _number_; metadata?: _MetricsMetadata_; numBackchannels: _number_; numInterruptions: _number_; numRequests: _number_; predictionDuration: _number_; timestamp: _number_; totalDuration: _number_; type: _"interruption_metrics"_; } how will I get this metrics? as metrics_collected is deprecated
t
Thanks — you’re asking how to obtain InterruptionMetrics now that 
metrics_collected
is deprecated. Short answer and recommended approach: • InterruptionMetrics are emitted when Adaptive Interruption Handling is enabled (default on LiveKit Cloud) and are available via per-plugin / per-session metrics listeners or from 
session.usage
at session end. • The pattern is to capture interruption metrics from the session’s metrics stream using the newer per-plugin listeners or a UsageCollector, then read session-level data at shutdown. Steps to implement: 1. Ensure adaptive interruption handling is enabled (or default on LiveKit Cloud) and VAD/aligned STT are active. 2. Add a per-plugin or session metrics listener to collect interruption metric objects as they arrive (they’ll contain fields like 
numInterruptions
, 
detectionDelay
, 
predictionDuration
, etc.). If your runtime provides a named event for plugin metrics, use that; otherwise use the updated metrics listener pattern your SDK exposes. 3. Aggregate during the session (use 
ModelUsageCollector
/ 
UsageCollector
or your own collector). 4. At session end, read 
session.usage
(specifically 
session.usage.model_usage
) or emit/post your aggregated InterruptionMetrics summary. If you tell me which SDK/language you’re using (agents-js, Python, etc.), I’ll provide a concise code example showing the exact listener and shutdown logging for InterruptionMetrics. Sources: Adaptive interruption handling | LiveKit Documentation, with interactive elements
a
how we can track it
t
I’ll show how to track InterruptionMetrics. Quick summary first, then next step. What to enable • Ensure Adaptive Interruption Handling is enabled (default on LiveKit Cloud). • Enable VAD and use an STT that supports aligned transcripts. How to capture metrics (recommended) 1. Attach a metrics listener for the session or the interruption plugin (use your SDK’s per-plugin metrics hook). This listener will receive InterruptionMetrics objects while the call runs. 2. Aggregate metrics as they arrive (use 
ModelUsageCollector
/ 
UsageCollector
or your own collector) to sum 
numInterruptions
, 
numBackchannels
, 
detectionDelay
, etc. 3. On session shutdown / room end, read 
session.usage.model_usage
and your aggregated interruption metrics and log or POST a final summary. Where the metrics appear • Live, via the per-plugin/per-session metrics listener events. • At end of call via 
session.usage
(session.usage.model_usage) and your own aggregated InterruptionMetrics summary. Needed from you to give code • Which SDK/language are you using (agents-js, Python, etc.) and do you want the final summary sent as a webhook or emitted in-room? Tell me that and I’ll provide a concise code example. , with interactive elements
a
Copy code
@session.llm.on("metics_collected")
def _on_metrics_collected(ev: MetricsCollectedEvent):
    metrics.log_metrics(ev.metrics, logger=logger)
this is not working
t
Thanks — I see your handler on 
session.llm
for 
"metics_collected"
isn't working. Quick fixes and the correct approach: • The 
metrics_collected
event on 
session.llm
is deprecated. Attach listeners at the session or per-plugin level instead (or use the runtime’s per-plugin metrics hook). • Also note your example has a typo: 
"metics_collected"
→ should be 
"metrics_collected"
(but still don’t attach it to 
session.llm
). Recommended Python pattern: 1. Attach to session-level metrics:
Copy code
def _on_metrics_collected(ev: MetricsCollectedEvent):
    # filter for the metric type you need, e.g., InterruptionMetrics
    metrics.log_metrics(ev.metrics, logger=logger)

# attach to session, not session.llm
session.on("metrics_collected", _on_metrics_collected)
1. If using realtime models or plugin-specific metrics, use the per-plugin listener pattern:
Copy code
def _sync_wrapper(ev: MetricsCollectedEvent):
    if isinstance(ev.metrics, RealtimeModelMetrics) or ev.metrics.type == "interruption_metrics":
        asyncio.create_task(_on_metrics_collected(ev))

session.on("metrics_collected", _sync_wrapper)
1. Aggregate during the session (UsageCollector or custom collector) and read 
session.usage.model_usage
on shutdown for a final summary. If you want, tell me: • which SDK and version (agents-js, Python SDK + version), and • paste the exact error or logs you see — and I’ll give a tailored, runnable snippet to fix your handler and capture InterruptionMetrics. Sources: Adaptive interruption handling | LiveKit Documentation, with interactive elements
a
Copy code
session.on("metrics_collected", _on_metrics_collected)
but this is deprecated
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
6 Views
Open in Slack
PreviousNext
Powered by