I have to use specific llms or something to get these to wor LiveKit Community #ask-ai

I have to use specific llms or something to get th...

elegant-television-86674

09/16/2025, 4:53 PM

I have to use specific llms or something to get these to work https://docs.livekit.io/agents/build/metrics/ i'm getting all 0 for the call metrics

tall-airline-16756

09/16/2025, 4:53 PM

Thanks for reaching out, Devin. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

tall-airline-16756

09/16/2025, 4:54 PM

It’s completely understandable to wonder why some metrics might show all 0s, so let me help clarify that for you: • Metrics such as freezes or RTT will show 0 if there were no issues during the call, which actually means everything went smoothly! • For AI Agents, if you see token counts or audio duration at 0, it usually means components like LLM, TTS, or STT weren’t used during that session. If you were expecting to see non-zero metrics, here are a few things to check: 1. Make sure media tracks are actively publishing and subscribing. 2. Confirm that client-side metrics reporting is turned on. 3. Ensure that agent components like LLM or TTS are being invoked during the session. Please feel free to reach out if you’d like me to walk you through verifying these settings or if you need any further assistance troubleshooting. I’m here to help! Sources: Logs, metrics, and telemetry | LiveKit Docs | livekit/livekit_internal.pb.go | Sources/LiveKit/Protos/livekit_models.pb.swift | livekit-plugins/livekit-plugins-google/livekit/plugins/google/beta/realtime...

elegant-television-86674

09/16/2025, 5:01 PM

if i'm using gpt, elevenlabs and deepgram i wouldnt be using these? For AI Agents, if you see token counts or audio duration at 0, it usually means components like LLM, TTS, or STT weren’t used during that session.

tall-airline-16756

09/16/2025, 5:01 PM

When you use GPT, ElevenLabs, and Deepgram services within a LiveKit agent, the metrics are automatically collected for you. To make sure everything is captured smoothly, here’s a little checklist: 1. Subscribe to the

metrics_collected

event in your agent session. 2. Use

metrics.logMetrics()

to log your metrics and

UsageCollector

to bring them all together. 3. Double-check that the plugins you want (like GPT, ElevenLabs, Deepgram) are actively being used during your session. If you’d like a hand setting up those event listeners or want to confirm your plugin usage, just let me know—I’m here to help! Sources: Logs, metrics, and telemetry | LiveKit Docs | examples/src/comprehensive_test.ts | metrics/metrics_stt.py

future-stone-4017

09/16/2025, 5:01 PM

How can I override the generate_reply function in AgentActivity class?

tall-airline-16756

09/16/2025, 5:02 PM

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!

elegant-television-86674

09/16/2025, 6:47 PM

this isnt working do you know why # LiveKit Metrics Collection: Implementation & Verification Reference: LiveKit metrics and UsageCollector docs: https://docs.livekit.io/agents/build/metrics/ ## Session wiring (STT, LLM, TTS, VAD) The agent session is created with all components and VAD turn detection so metrics are emitted for STT/LLM/TTS and EOU when used.

Copy code

python
# apps/open-source/agent/main.py
session = agents.AgentSession(
  stt=stt,
  llm=llm,
  tts=tts,
  vad=vad,
  turn_detection="vad",
  user_away_timeout=60,
)

## Subscribing to metrics_collected and aggregating usage We log each metrics object, feed it into UsageCollector, and add the values we need to our persisted call_metrics payload.

Copy code

python
# apps/open-source/agent/main.py
usage_collector = metrics.UsageCollector()

@session.on("metrics_collected")
def on_metrics_collected(ev: MetricsCollectedEvent):
  metrics.log_metrics(ev.metrics)
  usage_collector.collect(ev.metrics)

  # LLM tokens
  if hasattr(ev.metrics, 'llm') and ev.metrics.llm:
    call_metrics["llm_prompt_tokens"] += ev.metrics.llm.prompt_tokens or 0
    call_metrics["llm_completion_tokens"] += ev.metrics.llm.completion_tokens or 0

  # TTS chars
  if hasattr(ev.metrics, 'tts') and ev.metrics.tts:
    call_metrics["tts_character_count"] += ev.metrics.tts.characters_count or 0

  # STT user turn
  if hasattr(ev.metrics, 'stt') and ev.metrics.stt:
    call_metrics["user_turns"] += 1
    call_metrics["successful_completion"] = True

  # Count an agent turn when TTS produced speech
  if hasattr(ev.metrics, 'tts') and ev.metrics.tts and ev.metrics.tts.characters_count:
    call_metrics["agent_turns"] += 1

## Guaranteed fallback counting (speech events) If metrics events don’t fire (environment/version/reporting), we still count turns and TTS characters via speech commit events.

Copy code

python
# apps/open-source/agent/main.py
@session.on("user_speech_committed")
def _on_user_speech_committed(ev):
  call_metrics["user_turns"] += 1
  call_metrics["successful_completion"] = True

@session.on("agent_speech_committed")
def _on_agent_speech_committed(ev):
  call_metrics["agent_turns"] += 1
  if hasattr(ev, "text") and ev.text:
    call_metrics["tts_character_count"] += len(ev.text)

## Counting after every agent speak We also increment turns and TTS characters right after each successful session.say(...).

Copy code

python
# apps/open-source/agent/main.py
await session.say("hey this is Jess with Ghostify AI, how can I help you today?", allow_interruptions=False)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("hey this is Jess with Ghostify AI, how can I help you today?")

# Later when greeting
await session.say(greeting_text, allow_interruptions=True)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len(greeting_text or "")

Outbound agent uses the same pattern in all session.say(...) paths:

Copy code

python
# apps/open-source/agent/outbound-main.py
await session.say("I'm sorry, there is a configuration error and I can't save your information.")
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("I'm sorry, there is a configuration error and I can't save your information.")

await session.say("Thank you. Your information has been sent...", allow_interruptions=True)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("Thank you. Your information has been sent...")

await session.say("Hey there, thanks for reaching out to Ghostify. How's your day going?", allow_interruptions=True)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("Hey there, thanks for reaching out to Ghostify. How's your day going?")

## Persisting metrics and closing At session end (or on exception) we send the accumulated call_metrics structure to the API.

Copy code

python
# apps/open-source/agent/main.py
<http://logging.info|logging.info>(f":bar_chart: Sending call metrics for {ctx.job.id}")
await send_call_metrics(call_metrics, call_start_time)

## Backend summary endpoint used by dashboard To avoid route conflicts with /api/call-metrics/{call_id}, the dashboard reads aggregate stats from /api/metrics/summary.

Copy code

python
# apps/open-source/token-server/main.py
@app.get("/api/metrics/summary")
async def get_call_metrics_summary_safe():
  # computes total_calls, avg_duration, success_rate, avg turns, total tokens
  ...

Copy code

python
# apps-open-source/token-server/twilio_sync.py
calls = client.calls.list(start_time_after=since, page_size=50)
# upsert to twilio_calls, pull insights/recordings, enrich call_metrics best-match

elegant-television-86674

09/16/2025, 8:46 PM

Copy code

python
# apps/open-source/agent/main.py
session = agents.AgentSession(
  stt=stt,
  llm=llm,
  tts=tts,
  vad=vad,
  turn_detection="vad",
  user_away_timeout=60,
)

## Subscribing to metrics_collected and aggregating usage We log each metrics object, feed it into UsageCollector, and add the values we need to our persisted call_metrics payload.

Copy code

python
# apps/open-source/agent/main.py
usage_collector = metrics.UsageCollector()

@session.on("metrics_collected")
def on_metrics_collected(ev: MetricsCollectedEvent):
  metrics.log_metrics(ev.metrics)
  usage_collector.collect(ev.metrics)

  # LLM tokens
  if hasattr(ev.metrics, 'llm') and ev.metrics.llm:
    call_metrics["llm_prompt_tokens"] += ev.metrics.llm.prompt_tokens or 0
    call_metrics["llm_completion_tokens"] += ev.metrics.llm.completion_tokens or 0

  # TTS chars
  if hasattr(ev.metrics, 'tts') and ev.metrics.tts:
    call_metrics["tts_character_count"] += ev.metrics.tts.characters_count or 0

  # STT user turn
  if hasattr(ev.metrics, 'stt') and ev.metrics.stt:
    call_metrics["user_turns"] += 1
    call_metrics["successful_completion"] = True

  # Count an agent turn when TTS produced speech
  if hasattr(ev.metrics, 'tts') and ev.metrics.tts and ev.metrics.tts.characters_count:
    call_metrics["agent_turns"] += 1

## Guaranteed fallback counting (speech events) If metrics events don’t fire (environment/version/reporting), we still count turns and TTS characters via speech commit events.

Copy code

python
# apps/open-source/agent/main.py
@session.on("user_speech_committed")
def _on_user_speech_committed(ev):
  call_metrics["user_turns"] += 1
  call_metrics["successful_completion"] = True

@session.on("agent_speech_committed")
def _on_agent_speech_committed(ev):
  call_metrics["agent_turns"] += 1
  if hasattr(ev, "text") and ev.text:
    call_metrics["tts_character_count"] += len(ev.text)

## Counting after every agent speak We also increment turns and TTS characters right after each successful session.say(...).

Copy code

python
# apps/open-source/agent/main.py
await session.say("hey this is Jess with Ghostify AI, how can I help you today?", allow_interruptions=False)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("hey this is Jess with Ghostify AI, how can I help you today?")

# Later when greeting
await session.say(greeting_text, allow_interruptions=True)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len(greeting_text or "")

Outbound agent uses the same pattern in all session.say(...) paths:

Copy code

python
# apps/open-source/agent/outbound-main.py
await session.say("I'm sorry, there is a configuration error and I can't save your information.")
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("I'm sorry, there is a configuration error and I can't save your information.")

await session.say("Thank you. Your information has been sent...", allow_interruptions=True)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("Thank you. Your information has been sent...")

await session.say("Hey there, thanks for reaching out to Ghostify. How's your day going?", allow_interruptions=True)
call_metrics["agent_turns"] += 1
call_metrics["tts_character_count"] += len("Hey there, thanks for reaching out to Ghostify. How's your day going?")

## Persisting metrics and closing At session end (or on exception) we send the accumulated call_metrics structure to the API.

Copy code

python
# apps/open-source/agent/main.py
<http://logging.info|logging.info>(f":bar_chart: Sending call metrics for {ctx.job.id}")
await send_call_metrics(call_metrics, call_start_time)

## Backend summary endpoint used by dashboard To avoid route conflicts with /api/call-metrics/{call_id}, the dashboard reads aggregate stats from /api/metrics/summary.

Copy code

python
# apps/open-source/token-server/main.py
@app.get("/api/metrics/summary")
async def get_call_metrics_summary_safe():
  # computes total_calls, avg_duration, success_rate, avg turns, total tokens
  ...

## Twilio data enrichment (optional) - Sync: GET /api/twilio/sync?last_hours=24 - Tables: twilio_calls, call_recordings, Twilio columns on call_metrics - Matching heuristics: digits-only phone match, ±30 min window, duration closeness, room_name digits fallback. ```python # apps-open-source/token-server/twilio_sync.py calls = client.calls.list(start_time_after=since, page_size=50) # upsert to twilio_calls, pull insights/recordings, enrich call_metrics best-match

Open in Slack

Previous Next